Files
pxpipe/tests
teamchong e433b26caa feat(transform): lossless whitespace compactor on slab + reminder + tool_result
Production data (2026-05-20): real 161,101-char system slab being rejected
by the row-aware profitability gate as `not_profitable`. multiCol=2 packs
282 rows/image, but 2000+ newline-bounded lines need ~10 images, and at
2500 tok × 10 × 1.10 = 27,500 tok the gate (correctly) refuses since the
text baseline is cheaper.

Fix: strip per-line trailing whitespace and collapse 3+ consecutive
newlines to exactly 2 *before* feeding the renderer. Both transforms are
lossless for the model (whitespace at line-end carries no signal; blank-
line runs longer than 2 are visual noise). This drops visual-row count
without changing meaning, flipping borderline-rejected slabs profitable.

Accounting uses raw (pre-compaction) length for `origChars` and
`compressedChars`, so the savings credit reflects what Anthropic would
have billed — not the post-compaction figure.

Applied at four sites:
- combined static slab (system + tool docs)
- system-reminder block
- tool_result string content
- tool_result array-of-blocks (joined text)

8 new unit tests for `compactSlabWhitespace`: empty input, trailing
whitespace stripping, 3+ newline collapse, paragraph-break preservation,
indent preservation, idempotency, realistic markdown shrink.

270 tests pass, typecheck clean, build clean.
2026-05-20 00:07:42 -04:00
..