transform: compact JSON in tool docs (kills the sparse-fill crisis)

renderToolDoc pretty-printed every input_schema with 2-space indent,
putting each property on its own line. At cols=100 this wasted ~70% of
horizontal space — schemas rendered to many short rows instead of a few
dense ones.

Live measurement on 2026-05-19: 150 KB of pretty-printed tool docs in a
177 KB combined slab → 31 static-slab images per request at 40% fill.
With the new row-aware break-even gate (f0176c1), those rejected as
not_profitable — no savings on the user's typical workload.

Synthetic measurement on 30 realistic tool schemas:
- pretty: 320,843 chars / 14,074 lines → 100 images
- compact: 124,720 chars / 179 lines → 10 images
- 2.57× char shrink, 78× row shrink, 10× image cost reduction

Compact JSON is unambiguous: descriptions are already stripped under
compressSchemas, so only structural keys (type/properties/required/
enum/items) remain — those read fluently on one wrapped row. Frontier
models handle compact JSON natively; it's the default wire format for
tool_use blocks anyway.

Test pins the density gain on a representative schema.
This commit is contained in:
teamchong
2026-05-19 08:27:41 -04:00
parent f0176c1432
commit aed76dbdc2
2 changed files with 41 additions and 2 deletions
+16 -2
View File
@@ -736,12 +736,26 @@ function schemaHasStructure(schema: Record<string, unknown>): boolean {
return false;
}
/** Build the "## Tool: name\n<desc>\n<schema>" block for one tool definition. */
/** Build the "## Tool: name\n<desc>\n<schema>" block for one tool definition.
*
* Schema serialization is **compact** (no whitespace). Pretty-printing
* with 2-space indent was the dominant source of sparse fill: each schema
* key on its own line, indented, wastes 70%+ of horizontal space at
* cols=100. Live measurement on 2026-05-19 showed 150 KB of pretty
* tool-doc JSON across ~30 tools rendering to 31 static-slab images per
* request — a 40% fill ratio that pushed every request well past the
* break-even point.
*
* Compact form is still unambiguous JSON. Descriptions are already
* stripped under compressSchemas so only structural keys
* (type/properties/required/enum/items) remain — they read fluently
* on one line. Frontier models handle compact JSON natively (it's the
* default wire format for tool_use blocks). */
function renderToolDoc(t: ToolDef, includeSchema: boolean): string {
const parts: string[] = [`## Tool: ${t.name ?? '?'}`];
if (t.description) parts.push(t.description);
if (includeSchema && t.input_schema !== undefined) {
parts.push('```json\n' + JSON.stringify(t.input_schema, null, 2) + '\n```');
parts.push('```json\n' + JSON.stringify(t.input_schema) + '\n```');
}
return parts.join('\n');
}