Marks P0/P1/P2/P3/P4/P6 implemented and P5 deferred (with rationale:
pi does not do tool-pair summarization, and goose's emergency-vent
version is more likely to amplify brain rot than reduce it on the
long-context models where it would fire latest).
Signed-off-by: Michael Neale <michael.neale@gmail.com>
The code_execution sandbox encourages the model to bundle multiple
sub-tool calls and return one labeled object like
'Result: { foo, bar, baz }'. Without per-field truncation, a single fat
field (full source file, build log, large query result) bloats the
whole bundle even when the other fields are tiny and useful.
Add truncate_execute_output() which intercepts pctx_code_mode's
ExecuteOutput just before output.markdown() is called and:
- caps stdout and stderr if they exceed the per-field cap,
- walks the JSON value tree and truncates any leaf string
longer than the cap with a marker.
Truncation is UTF-8 safe and recursive (nested objects + arrays). The
top-level P0 cap (16 KB on the whole tool_result text) still applies as
a backstop; this P2 cap (8 KB per field by default) operates earlier so
that one large field can't drown out the small useful ones in the same
bundle.
- Defaults: 8192 bytes per string field.
- Tunable: GOOSE_CODE_EXEC_FIELD_MAX_BYTES (0 to disable).
- Applied to both handle_execute_typescript and handle_execute_bash
(execute_bash also wraps a bash invocation whose stdout/stderr can be
large; same treatment).
Three serial unit tests cover (a) leaf-only truncation across object,
array, and nested object, (b) end-to-end ExecuteOutput cap, (c) env
disable.
Signed-off-by: Michael Neale <michael.neale@gmail.com>
Three coordinated changes to make compacted-state sessions less prone to
confabulation.
(1) Rewrite the compaction summary template (prompts/compaction.md)
Previously the summarizer was told to produce a freeform <analysis> blob.
Replace with an explicit section-based template (Goal / Constraints /
Progress (Done|In Progress|Blocked) / Key Decisions / Critical Context /
Next Step) and trailing <read-files>/<modified-files> blocks. Tells the
model that the summary is its only memory, to prefer concrete facts over
prose, and to not invent facts that aren't in the transcript.
(2) Extract file history from the transcript
New extract_file_history() walks ToolRequest content and pulls 'path'
arguments out of read/write/edit-shaped tool calls (read, write, edit,
file_read, view, text_editor, str_replace_editor, with or without
'<server>__' prefixes; the suffix after the last '__' is also matched).
Results are deduplicated in first-seen order. The lists are threaded
into SummarizeContext as read_files and modified_files and rendered in
the template via {% if read_files %} / {% if modified_files %} blocks.
The summarizer sees these hints and includes them in its
<read-files>/<modified-files> output. Cheap and accurate; if a tool
shape doesn't match, the section just stays empty.
(3) Honest continuation prompts
The three continuation messages (auto compact, tool-loop compact, manual
/compact) previously instructed the model 'Do not mention that you read
a summary or that conversation summarization occurred.' That's a direct
confabulation prompt. Replace with text that tells the model the
previous message is a summary, that the original messages are gone, and
that it should say so plainly and ask rather than guess when something
isn't covered.
Tests:
- 3 new unit tests in context_mgmt covering extract_file_history (reads
vs writes, dedup, ignoring shell/tree).
- 1 new unit test asserting all three continuation prompts mention
'summary' and do NOT contain 'Do not mention'.
- compaction.rs integration tests updated: MockCompactionProvider's
is_compaction heuristic broadened to match 'summary' as well as
'summarize' (the new template prose uses both); the continuation-text
detector in the assertion broadened correspondingly. All 3
integration tests pass.
Signed-off-by: Michael Neale <michael.neale@gmail.com>
The upstream pctx_code_mode tool descriptions are explicit attractors
toward 'bundle multiple sub-tool calls and return one fat object'. That
intent is fine for round-trip count but creates context bloat when any
bundled field carries a large body (file dumps, build output, edit
echoes).
Add a goose-side 'CONTEXT HYGIENE' addendum appended to each of the
three execute_typescript descriptions (catalog, filesystem, sidecar)
that:
- tells the model to bundle only when each field is small (~5 KB),
and to call large-output tools on their own,
- tells it to return only the fields it actually needs,
- tells it to use line/limit/head/tail/grep instead of cat <big-file>.
Upstream descriptions are kept verbatim and the goose advice is
appended, so callers reading upstream guidance get the upstream contract
plus the goose context-management override.
The addendum is plain prose with no schema changes; no tool semantics
change. One unit test covers the prefix-preserving combination.
Signed-off-by: Michael Neale <michael.neale@gmail.com>
Restores the 'read' tool on DeveloperClient. The handler was removed in
f2ad0a852 because ACP clients delegate filesystem I/O and intercept
'read' at the AcpTools layer. Non-ACP sessions were left without a
structured read tool, forcing the model into shell({command:'cat ...'})
which has no line/limit knobs.
Changes:
- 'read' tool registered on DeveloperClient with FileReadParams schema.
- DEFAULT_FILE_READ_LIMIT = 2000 lines applied when caller passes no
'limit'. When truncation occurs, the result body is followed by:
[goose: showing lines X-Y of Z (N more lines not shown). Pass
'line' and/or 'limit' to read a different range.]
- developer_instructions text on both POSIX and Windows updated to tell
the model to prefer 'read' over 'cat'/'sed' / 'type'/'Get-Content'.
- ACP layer is unchanged: when fs_read is enabled, AcpTools still
intercepts 'read' and routes to the ACP client; otherwise it falls
through to the developer impl with the new cap.
Tests updated: previous read tests asserted exact untruncated bodies;
they're replaced with three tests covering (a) small file no marker,
(b) explicit range emits marker, (c) default cap truncates a 5000-line
file at line 2000 with marker.
Signed-off-by: Michael Neale <michael.neale@gmail.com>
Walks the conversation snapshot just before it's handed to the provider
and replaces any tool_response text block whose (tool_name, arguments,
body) triple is byte-identical to a prior call's result with a short
pointer back to the first occurrence.
- Same file read twice with the same args -> second copy becomes
'[goose: identical to tool result at turn N (sha=...), N bytes elided,
tool=read]'.
- Bodies under 512 bytes are skipped (not worth the marker).
- Errors (Err arm) are never deduped.
- Differing args or differing body => pass-through, so genuine changes
flow normally.
- Session DB and on-screen view keep the full content; only the LLM
payload is deduped.
- Disable via GOOSE_TOOL_RESULT_DEDUP=0.
6 unit tests covering: dedup fires, args differ -> no dedup, body
changed -> no dedup, small bodies skipped, env disable, and third copy
points to the first occurrence (not the second).
Signed-off-by: Michael Neale <michael.neale@gmail.com>
Adds a per-text-block size cap on every tool_result that flows back to
the model, applied at the single MessageContent::tool_response{,_with_metadata}
choke point.
- Default cap: 16 KB per text block.
- Tunable via GOOSE_TOOL_RESULT_MAX_BYTES; set to 0 to disable.
- Truncates at a UTF-8 boundary and appends a marker telling the model
exactly how to retrieve more (narrower line/limit, grep, head, etc.).
- Non-text content (images, audio, resources) is untouched.
- Tests cover under-cap pass-through, over-cap truncation, env-zero
disable, and UTF-8 boundary correctness. Env-touching tests are
serialized via serial_test.
Companion BRAINROT.md explains the assessment, mechanisms, and
followup improvements (P1 dedup, P2 code_execution per-field cap,
P3 prompt softening, P4 structured compaction, P5 better tool-pair
summarization, P6 lean read tool).
Signed-off-by: Michael Neale <michael.neale@gmail.com>
Michael Neale