teamchong 440ca7f448 test: real-shape regression fixtures from production events.jsonl ...

Fragility #3 from end-of-session audit. The previous gate tests pinned
isCompressionProfitable(slab, 100, 2, 2.5) === true against 'A'.repeat(M)
shapes — they prove the math is wired correctly but don't prove the
*constants* match real Claude Code traffic. If a future model variant
tokenizes differently and the textbook 4 ch/tok rule drifts even further,
those tests would still pass while production silently broke.

Anonymized 6 production shapes from events.jsonl (2026-05-19..2026-05-20):
- PRODUCTION_SLAB_161K (multi-col, 52 chars/row, 11 ings/141 lines/2 cols)
- PRODUCTION_SLAB_135K_DENSE (neuline-heavy, 19 chars/row, 24 ings)
- PRODUCTION_SLAB_169K_HEAVY (very dense, 16 chars/row — actually rejected)
- BELOW_MIN_CHARS_TINY (142 chars, pre-filter)
- BELOW_MIN_CHARS_BORDERLINE (1123 chars, pre-filter)
- HISTORY_COLLAPSED_LONG_SESSION (collapsed path captured)

Each fixture stores its real captured decision plus the shape parameters
(origChars, numCols, approxCharsPerRow). synthesizeText() pads 'A'.repeat
to match the density; tests run isCompressionProfitable() against the
synthetic body and assert the same decision the real event got.

\#\# Notable: 135K_DENSE divergence

The real 135K neuline-heavy event was ACCEPTED (compressed=true) but the
synthetic 'A'.repeat(19) reconstruction REJECTS even at cpt=2.5. Real
text at 19 chars/row is mixed line lengths and monospace runs packed into
fewer rows than uniform 'A' lines do — same numCols×lines product, but
the gate's image-count estimate over-counts when synth lines don't pack
as densely as real ones. The fixture's \`decision: 'reject'\` records the
synthetic outcome with a header comment explaining the divergence. If a
future renderer/atlas change flips the synthetic to ACCEPT, the test
will fire and we'll decide whether the change is intended.

\#\# Export changes

SLAB_CHARS_PER_TOKEN and HISTORY_CHARS_PER_TOKEN exported from transform.ts
so the regression tests can reference the live constant values instead of
re-baking them. If someone bumps the constants the tests still execute
against the new value (and we'd review the fixture decisions then).

\#\# Tests

8 files / 280 tests pass, typecheck clean, build clean. 7 new it() blocks
cover the 5 captured shapes + the synthetic divergence note.

2026-05-20 10:50:16 -04:00

..

real-shapes.ts

test: real-shape regression fixtures from production events.jsonl

2026-05-20 10:50:16 -04:00