Adds a third tee branch in createProxy that walks the response body and
counts Unicode code units across text_delta / thinking_delta /
input_json_delta / tool_use blocks, plus a redacted_thinking block count.
Same shape on streaming (SSE) and non-stream (content[]) responses.
Numbers are independent of Anthropic's usage.output_tokens - they give a
real ruler against the redacted_thinking-inflated bill that surfaced in
the May-2026 weekly-meter audit (#22).
Wiring:
OutputMeasurement type in proxy.ts (textChars, thinkingChars,
toolUseChars, redactedBlockCount)
teeForUsage now returns measurementPromise alongside usagePromise +
errorBodyPromise; Promise.all in handle() awaits all three
ProxyEvent gains optional measurement: OutputMeasurement
TrackEvent gains text_chars_measured / thinking_chars_measured /
tool_use_chars_measured / redacted_block_count_measured (optional,
non-breaking JSONL shape)
toTrackEvent() copies them through
Tests (6 new in proxy-usage.test.ts):
text_delta count across multi-event SSE
message_delta output_tokens overrides message_start placeholder
thinking_delta + redacted_thinking block count
tool_use input_json_delta reassembly
non-stream content[] walks same shape
5xx no-body path leaves measurement undefined
288 tests pass, tsc --noEmit clean, dist/node.js builds.
Dashboard surfacing lands in a follow-up so the wire-up stays reviewable.
Refs #22
The fifth headline card surfaces the absolute number Anthropic's weekly
meter actually sees: input + output×5 in input-token-equivalents. Pairs
with the baseline counterfactual in the sub-line so a heavy-output day
shows as a real spike instead of hiding behind a flat 73% gauge.
Grid CSS bumps to 5 columns with a middle breakpoint at 1200px so the
cards don't crush at typical widths. No new computation — both fields
already lived in the /proxy-stats payload from #20.
The old single-rate weight (cr>0 ? 0.10 : cc>0 ? 1.25 : 1.0) collapsed
the entire cacheable prefix to one class. On warm turns the proxied path
bills cc * 1.25 for the new user-typed tail AND cr * 0.10 for the rest
of the prefix - but the formula attributed 100% to cr * 0.10, making the
unproxied path look ~12.5x cheaper than reality and producing negative
'saved' on every warm row.
The corrected per-event break-even (no compression, identical content):
cold start (cr=0, cc>0): ccU = cacheable, crU = 0
warm turn (cc>0, cr>0): ccU = min(cc, cacheable), crU = cacheable - ccU
no caching (cc=0, cr=0): ccU = 0, crU = 0
cache-read (cr>0, cc=0): ccU = 0, crU = cacheable
baseline_eff = ccU*1.25 + crU*0.10 + coldTail*1.0
Centralized as computeBaselineInputEff() and computeActualInputEff() in
src/core/baseline.ts, used by the live dashboard (dashboard.ts), the
JSONL replay path (dashboard.ts), and per-session rollup (sessions.ts).
Verified on the May-2026 regression (7 events, mixed cc/cr, not negative):
pre-fix sum = -9,786 'saved' tokens, post-fix sum = +19,452 tokens,
matching the per-event compression delta (~+2,780/event) exactly.
Aggregate impact on the 914-event corpus: 'saved %' moves from 73.1% to
73.2% - the headline barely shifts because warm turns are a minority,
but per-event dashboard rows no longer show false negatives.
282 tests pass, tsc --noEmit clean.
- Headline: "pixels instead of tokens" → "pixels instead of text" (images
bill as tokens too — the honest framing is pixels vs text)
- Removed the redundant "Why this works in 2026" blockquote since the
Opus 4.7 explanation already lives in the intro paragraph
The intro now leads with the hypothesis (pixels pack more semantic info
per unit of context than serialized text) and treats the token reduction
as measurable evidence rather than the headline.
Adds a "What this is NOT" section to address the ToS / billing-loophole
read directly:
- Not ToS evasion (uses the documented vision API as documented)
- Not a billing loophole (savings hold even if Anthropic re-prices)
- Not cost arbitrage (cost is a side effect of density)
Same numbers, same code, different stance: defensive about the framing
("I'm not paying for churn I didn't cause") rather than offensive about
the pricing.
Anthropic's /v1/messages usage block returns more fields than we were
capturing. The flat 4-field view (input_tokens, output_tokens,
cache_creation_input_tokens, cache_read_input_tokens) historically dropped:
- cache_creation.ephemeral_5m_input_tokens (1.25x rate price tier)
- cache_creation.ephemeral_1h_input_tokens (2x rate price tier)
- server_tool_use.web_search_requests (billed per-request, not per-token)
Extends:
- Usage interface (types.ts): nested cache_creation + server_tool_use
- TrackEvent (tracker.ts): cache_create_5m_tokens, cache_create_1h_tokens,
web_search_requests as optional fields
- toTrackEvent copy logic: passes through when defined, omits when absent
(a real 0 stays 0 - the dashboard needs to tell "missing" from "zero")
Also adds .gitignore entries for draumode MCP scratch files
(*.draumode.ts, *.excalidraw) so exploratory diagrams don't leak into
the repo.
Pure data collection. No behavior change. Sets up the dashboard fixes
(tasks #20 and #21) where the data will actually get rendered.
282 tests pass, typecheck clean, build clean.
Fragility #3 from end-of-session audit. The previous gate tests pinned
isCompressionProfitable(slab, 100, 2, 2.5) === true against 'A'.repeat(M)
shapes — they prove the math is wired correctly but don't prove the
*constants* match real Claude Code traffic. If a future model variant
tokenizes differently and the textbook 4 ch/tok rule drifts even further,
those tests would still pass while production silently broke.
Anonymized 6 production shapes from events.jsonl (2026-05-19..2026-05-20):
- PRODUCTION_SLAB_161K (multi-col, 52 chars/row, 11 ings/141 lines/2 cols)
- PRODUCTION_SLAB_135K_DENSE (neuline-heavy, 19 chars/row, 24 ings)
- PRODUCTION_SLAB_169K_HEAVY (very dense, 16 chars/row — actually rejected)
- BELOW_MIN_CHARS_TINY (142 chars, pre-filter)
- BELOW_MIN_CHARS_BORDERLINE (1123 chars, pre-filter)
- HISTORY_COLLAPSED_LONG_SESSION (collapsed path captured)
Each fixture stores its real captured decision plus the shape parameters
(origChars, numCols, approxCharsPerRow). synthesizeText() pads 'A'.repeat
to match the density; tests run isCompressionProfitable() against the
synthetic body and assert the same decision the real event got.
\#\# Notable: 135K_DENSE divergence
The real 135K neuline-heavy event was ACCEPTED (compressed=true) but the
synthetic 'A'.repeat(19) reconstruction REJECTS even at cpt=2.5. Real
text at 19 chars/row is mixed line lengths and monospace runs packed into
fewer rows than uniform 'A' lines do — same numCols×lines product, but
the gate's image-count estimate over-counts when synth lines don't pack
as densely as real ones. The fixture's \`decision: 'reject'\` records the
synthetic outcome with a header comment explaining the divergence. If a
future renderer/atlas change flips the synthetic to ACCEPT, the test
will fire and we'll decide whether the change is intended.
\#\# Export changes
SLAB_CHARS_PER_TOKEN and HISTORY_CHARS_PER_TOKEN exported from transform.ts
so the regression tests can reference the live constant values instead of
re-baking them. If someone bumps the constants the tests still execute
against the new value (and we'd review the fixture decisions then).
\#\# Tests
8 files / 280 tests pass, typecheck clean, build clean. 7 new it() blocks
cover the 5 captured shapes + the synthetic divergence note.
Pre-4.7 vision couldn't reliably OCR dense monospace glyphs - errors
would corrupt the prompt before the model read it. Opus 4.7 bumped
the long-edge image cap 1568->2576px (3.3x pixels) and document-OCR
benchmarks (DocVQA 87->94%, ChartQA 80->88%) per Anthropic's release
notes.
Pixelpipe still renders at 1568x1568 - this is a model fidelity
change, not a renderer change. But it's why this approach works now
when it didn't a year ago.
Cites anthropic.com/news/claude-opus-4-7.
The override-gate previously used `o.charsPerToken !== CHARS_PER_TOKEN`
to detect "host pinning". After CHARS_PER_TOKEN=4 was replaced with
slab/history-specific 2.5 constants, a host that deliberately passes the
literal `4` (the same as DEFAULTS) was silently swapped to 2.5 - the
gate couldn't distinguish "host passed nothing" from "host passed 4".
Fix: check the raw opts (\`opts.charsPerToken !== undefined\`) instead
of comparing the merged value against a sentinel. Host-supplied `4` now
flows through as `4`, even though it matches the default.
Tests pin both directions:
- slab path (render.test.ts): explicit cpt=4 keeps the 161k slab
REJECTED at cpt=4 (would be ACCEPTED at the built-in 2.5)
- history path (history.test.ts): explicit cpt=4 keeps a 14-turn closed
conversation REJECTED at cpt=4 (would collapse at the built-in 2.5),
with companion cpt=2.5 test proving the fixture actually straddles
275 tests pass, typecheck clean, build clean.
Proxy targets Opus 4.7. Input pricing has been flat across Opus 4.5/4.6/4.7
($5/$25 per MTok), so the historical Opus 4 column ($15/$75) was framing
noise unrelated to actual cost.
~80-line section covering the 4-breakpoint cache cliff that history
collapse addresses, the closed-prefix walk, the 4 honesty gates, and
today's HISTORY_CHARS_PER_TOKEN=2.5 fix wired in b563751.
Links to the live data point in events.jsonl (12:30:01 event,
collapsed_turns=175, collapsed_chars=180,684) and the relevant
transform.ts / core/history.ts call sites.
Addresses GHSA-vmjp-67rh-pgjc (undici via wrangler <4.x).
Major jump 3 → 4, but wrangler.toml schema unchanged
(compatibility_date, compatibility_flags, vars all still valid)
and build/typecheck/test all clean.
Held at 4.92.0 (4.93.0 is inside the 3-day minimumReleaseAge gate).
Earlier commit 26fbb67 "swap to pnpm-lock.yaml as source of truth"
untracked package-lock.json but pnpm-lock.yaml never made it in —
a global ~/.gitignore excludes pnpm-lock.yaml across all repos. The
lockfile was silently un-tracked the whole time.
Fix: add !pnpm-lock.yaml to the repo's .gitignore to override the
global rule, then stage the lockfile.
Verified: git check-ignore now reports .gitignore:11:!pnpm-lock.yaml.
Addresses GHSA-9crc-q9x8-hgqq, GHSA-jcfh-c3jc-r2q6 (vitest <2.1.9).
Vitest 4 requires Vite 6+, so add vite as an explicit dev dep
(was previously transitive). 273 tests still pass clean.
Addresses GHSA-67mh-4wv8-2f99 (esbuild dev server allowed any website
to send requests and read responses, patched in ≥0.25.0). Dev-only —
we use esbuild as the bundler in scripts/build.mjs, never as a server.
Build, typecheck, and 273 tests all clean.
Refuses to install any package version published less than 3 days ago.
Catches the typical malicious-package lifecycle (publish → discovered →
unpublished within hours/days). Legitimate security patches also wait
3 days; deliberate trade-off for a manually-updated repo.
The repo uses pnpm (packageManager pinned in package.json, pnpm-lock.yaml
tracked, pnpm-workspace.yaml present) but package-lock.json was also
tracked from an earlier npm install. Dependabot reads stale npm metadata
from it that doesn't match what pnpm actually installs.
- Untrack and delete package-lock.json
- Add it to .gitignore so a stray npm install can't reintroduce it
Previous version only listed system+history+tool_result. Real input
shape also includes:
- tools field (built-in schemas + every MCP server's tool definitions
- usually the largest single bucket; grows with each MCP server
wired up)
- skills loaded into the system prompt (SKILL.md per skill)
- subagent definitions
- <system-reminder> blocks Claude Code injects into user messages
Headline math unchanged - all of these land in the same static slab or
closed-prefix history that pixelpipe image-compresses.
Production telemetry (N=10 events with history_reason="not_profitable",
2026-05-18..05-20) shows body-level chars-per-token on rejected history
events: median 1.09, max 1.10. The gate at cpt=4 was estimating text as
3.7× cheaper than reality on every history-collapse decision in the
sampled traffic — same shape of bug as the slab path fixed in c36ec3f.
Why this matters more than the slab fix:
• The slab is cached (cache_create×1.25 once, cache_read×0.10 after).
Slab compression saves bytes mostly on cold misses + 2k tok/warm hit.
• History content is NOT cached. Once turns leave the cache breakpoint
budget (4 max per request), every subsequent request pays 100% for
them. A long agentic session re-bills the same closed-prefix turns
on every turn — exactly the workload Variant C was designed to
compress, and exactly the path the gate was silently rejecting.
Apply same fix as c36ec3f: HISTORY_CHARS_PER_TOKEN=2.5 (upper bound of
empirical data, identical safety property to SLAB_CHARS_PER_TOKEN since
history content shape ⊆ slab content shape in tool-heavy sessions).
Host can still override via TransformOptions.charsPerToken. The
override-gate `o.charsPerToken !== CHARS_PER_TOKEN` means unspecified
requests (DEFAULTS.charsPerToken=4) get the empirical cpt while explicit
non-default host values win — same intentional pattern as the slab path.
Test: regression fixture pinning a 14-turn × 1500 chars/turn conversation
where:
• cpt=4 path → 'not_profitable' rejection (image cost 5k > text 4k)
• cpt=2.5 path → 'collapsed' with 10 turns merged (text 6.4k > image 5k)
Counterfactual half uses cpt=4.5 to bypass the override-gate (passing
exactly 4 collides with the DEFAULTS sentinel).
273 tests pass, typecheck clean, build clean.
Production data (N=354 cold-miss count_tokens probes, 2026-05-18..05-20):
real body-level chars-per-token is 1.17 median, 2.62 max — never near
the English-prose 4 baked into CHARS_PER_TOKEN. The gate has been
estimating text-token cost at 3.4× cheaper than reality and silently
rejecting every realistic Claude Code system slab.
Concrete production case (orig_chars=161,101, multi-col=2):
• OLD gate (cpt=4): text = 161101/4 = 40,275 tok
image = 8 × 5500 = 44,000 tok
→ reject (margin: -3,726)
• NEW gate (cpt=2.5): text = 161101/2.5 = 64,440 tok
image = 8 × 5500 = 44,000 tok
→ ACCEPT (margin: +20,440)
• Reality (cpt=1.17): text = 137,694 tok, savings = 93,694 tok/request
SLAB_CHARS_PER_TOKEN=2.5 is the upper bound of observed cpt across the
sample — picking the upper bound keeps the prime-directive guarantee
intact: the text-token estimate is a LOWER bound on real text cost, so
any `imageCost < textTokens` decision is also `imageCost < realTextCost`
for any future workload with cpt ≤ 2.5.
Scope: this is slab-specific. Reminders and tool_result content have
unknown shape (could be raw English prose with cpt~4), so those gate
call sites still use the conservative CHARS_PER_TOKEN=4. Host can
override per-request via TransformOptions.charsPerToken (e.g., to plug
a live empirical fit in front of the static default).
Tests: 2 new regression tests pinning (a) the production-shape 161k slab
compresses end-to-end without an explicit cpt override, and (b) the gate
math at cpt=2.5 vs cpt=4 produces accept-vs-reject on the same input.
272 tests pass, typecheck clean, build clean.
Production data (2026-05-20): real 161,101-char system slab being rejected
by the row-aware profitability gate as `not_profitable`. multiCol=2 packs
282 rows/image, but 2000+ newline-bounded lines need ~10 images, and at
2500 tok × 10 × 1.10 = 27,500 tok the gate (correctly) refuses since the
text baseline is cheaper.
Fix: strip per-line trailing whitespace and collapse 3+ consecutive
newlines to exactly 2 *before* feeding the renderer. Both transforms are
lossless for the model (whitespace at line-end carries no signal; blank-
line runs longer than 2 are visual noise). This drops visual-row count
without changing meaning, flipping borderline-rejected slabs profitable.
Accounting uses raw (pre-compaction) length for `origChars` and
`compressedChars`, so the savings credit reflects what Anthropic would
have billed — not the post-compaction figure.
Applied at four sites:
- combined static slab (system + tool docs)
- system-reminder block
- tool_result string content
- tool_result array-of-blocks (joined text)
8 new unit tests for `compactSlabWhitespace`: empty input, trailing
whitespace stripping, 3+ newline collapse, paragraph-break preservation,
indent preservation, idempotency, realistic markdown shrink.
270 tests pass, typecheck clean, build clean.
Rip the compressHistory opt-in and run Variant C history-image collapse on
every request that has a closed-prefix run. There is now ONE codepath: no
toggle, no env switch, no per-request opt-in. The reason the option was
defaulted-off was a -250% live measurement on 2026-05-19 (commit c3506e3 era);
that wasn't a "marginal upside" problem, it was a math bug. This commit
fixes the bug and turns the feature on.
Root cause of the -250% loss
----------------------------
`isCompressionProfitable()` has two paths: a row-aware path (string input,
matches `renderTextToPngs` image budgeting exactly) and a looser chars-only
path (number input, assumes dense lines). History.ts was calling it with
`text.length` (number) → loose chars-only estimate. History text is
newline-heavy (`--- role ---` headers, JSON arg blocks, [tool_use] /
[tool_result] labels), so the chars-only estimate under-counted images by
5-10× and let net-losers through the gate.
Secondary contributor: tool_use args were serialised with
`JSON.stringify(input, null, 2)` — pretty-printed, one short row per JSON
field. At cols=100 each field claimed a full row instead of wrapping at
~100 chars, inflating image count for tool-heavy histories.
Changes
-------
- src/core/history.ts
- `ProfitableFn` signature: `(text: string, cols: number)` (was `textLen: number`).
Documented the asymmetry so callers can't regress this.
- `blocksToText` tool_use: compact `JSON.stringify(input)`, no indent.
- Pass `text` (not `text.length`) into the profitability check.
- src/core/transform.ts
- Delete `compressHistory`, `historyKeepTail`, `historyMinPrefix` from
TransformOptions and DEFAULTS.
- History collapse is now unconditional when `messages[]` is non-empty.
- Wrap the gate in a closure that pins `numCols=1` + per-request `cpt`,
so the gate's row-aware decision is byte-identical to the single-col
renderer's actual image count.
- src/worker.ts: drop `COMPRESS_HISTORY`, `HISTORY_KEEP_TAIL`,
`HISTORY_MIN_PREFIX` env vars from `Env` + transform options.
- src/node.ts: stale comment refresh — no toggles, period.
- tests/history.test.ts:
- Rename describe "transformRequest + compressHistory" →
"transformRequest history compression (always-on)".
- Remove every `compressHistory: true, historyKeepTail: X, historyMinPrefix: Y`
from transformRequest calls — options no longer exist.
- Bump fixtures for default `keepTail=4` / `minCollapsePrefix=10`
(14-turn body, 3500 chars/turn) so the now-honest row-aware gate
accepts. Renamed "8-closed + 2-live" → "10-closed + 4-live" to match
the new defaults.
- Bump straddle-test per-turn body 2500→3500 chars: tool block labels
add ~65 chars of header overhead that pushed the tighter fixture
under the row-aware boundary.
- Rename "JSON-pretty args" test → "COMPACT JSON args"; assert no
pretty indentation.
Verification
------------
- npm test → 263/263 green
- npm run typecheck → clean
- npm run build → dist/node.js builds clean
Production runs multiCol=2 by default. The pre-flight gate
`isCompressionProfitable()` was using a flat TOKENS_PER_IMAGE=2500
constant at every numCols, so it believed multi-col images were ~2x
cheaper than they actually are and would let net-loss compressions
through on wider canvases.
Replace the flat constant with `effectiveTokensPerImage(numCols)`:
numCols=1: 2500 (empirical, N=33 cold-miss measurement)
numCols>1: 2500 * n * 1.10 (linear extrapolation + 10% margin)
The 10% margin absorbs extrapolation noise — we don't yet have
empirical cost measurements at numCols>=2. Per user constraint
"ok pre-flight gate is ok to be not correct as long as we don't
lose money," every uncertainty resolves in favor of pass-through.
At worst we miss opportunities; we never burn money on net-loss
images.
Test fixtures bumped to match the new break-even (~22k chars at
n=2). Two obsolete cpt-flip tests REWRITTEN to assert the safe
behavior: cpt override flips at numCols=1, but at numCols=2 the
safety margin holds and the slab stays rejected.
Old break-even at default cpt=4:
n=1: ~10k chars (1 image @ 2500 tok == 10k/4 = 2500 text tok)
n=2 [old/wrong]: ~10k chars (same flat cost)
n=2 [new/safe]: ~22k chars (1 image @ 5500 tok == 22k/4 text tok)
Verified: 263/263 tests pass, typecheck clean, build clean.
The baseline_tokens probe alone reports the COLD cost of the unproxied
body — but the user's request actually carries `cache_control` markers
that would have cached on the unproxied path too. Comparing cold-
baseline against warm-actual was an apples-to-oranges inflation of
saved%.
Fix: fire a SECOND count_tokens probe on the body TRUNCATED at the last
`cache_control` marker. Difference between the two is (cacheable_prefix,
cold_tail). Dashboard weights the prefix by the SAME cache class the
actual response landed in:
cacheable = baseline_cacheable_tokens or 0
cold_tail = baseline_tokens − cacheable
weight = cr>0 ? 0.10 (warm hit — both paths cache)
: cc>0 ? 1.25 (cold create — both paths build cache)
: 1.0 (no caching — marker rejected or absent)
baseline_input_eff = cacheable·weight + cold_tail·1.0
actual_input_eff = input + cache_create·1.25 + cache_read·0.10
saved = baseline_input_eff − actual_input_eff
Now the headline number is the exact unproxied counterfactual, not a
cold-every-time approximation. No estimation, no α, no assumptions —
every input is a measurement.
Wiring:
- src/core/proxy.ts: buildCountTokensCacheablePrefix() walks messages
backward → system backward → tools backward (reverse cache order) for
the last cache_control marker, truncates, attaches a sentinel user
message when needed. Second probe fires in parallel; null/missing
leaves the field absent.
- src/core/transform.ts + tracker.ts: new baselineCacheableTokens /
baseline_cacheable_tokens fields, emitted only when > 0.
- src/dashboard.ts: per-event cache-aware math in update() and
replay(); Totals renamed to baselineInputWeighted; /proxy-stats
payload field is `baseline_input_weighted`.
- src/sessions.ts: per-session tokensSavedEst uses the same formula so
session and global rollups agree.
Tests: 2 new in proxy-usage covering (a) body with cache_control →
two probes fire with different bodies and both numbers land on info,
(b) cacheable-prefix probe failure doesn't poison the full probe.
260→262 tests, all green; typecheck and build clean.
The headline saved%/saved$ numbers are now MEASURED, not estimated:
baseline = sum of /v1/messages/count_tokens(originalBody).input_tokens
actual = sum of (input + cache_create×1.25 + cache_read×0.10)
saved = baseline − actual
Output is excluded from both sides — it's identical with or without
compression (same prompt → same response), so it cancels in the
numerator and would only drag the percentage toward zero. The proxy
now makes one parallel side call to /v1/messages/count_tokens (free,
no billing) before forwarding /v1/messages, and the result lands on
TransformInfo.baselineTokens → TrackEvent.baseline_tokens → the
DashboardState rollup.
Removed (~280 lines of dead code):
- FALLBACK_ALPHA, FALLBACK_ALPHA_LOW, FALLBACK_ALPHA_HIGH constants
- estImageTokens(), baselineCost(), effectiveCost() helpers
- FitSample interface + FIT_SAMPLE_CAP + fitSamples ring buffer
- DashboardState.fitCosts() (~218 lines of OLS regression)
- Three-mode ladder (joint / constrained / stale) and all UI for it
- saved_pct_low/high, saved_effective_tokens_low/high range fields
- renderSavingsMath() range/regime branches
- m_pct_regime DOM node + its color/text wiring
- cost_fit payload field + all frontend consumers
/proxy-stats payload is now: requests, compressed_requests,
baseline_input_tokens, actual_input_weighted, saved_input_tokens,
saved_pct, saved_usd, pricing_assumptions, uptime_sec,
compression_enabled. Nothing else.
The recent-requests table now shows baseline / actual / saved per
row (input-side, matching the cards) instead of the old img-tok
estimate + effective_actual (which mixed output × 5 in inconsistently).
Sessions panel: per-session tokensSavedEst now uses the same real
baseline−actual formula. Events without both a count_tokens probe
AND an upstream usage block don't contribute to the rollup — no
synthetic 4-char/tok fallback.
Tests:
- dashboard-fit.test.ts: deleted (fitCosts is gone)
- proxy-usage.test.ts: inverted the privacy regression — count_tokens
is now explicitly expected as the one whitelisted side path; added
baselineTokens-attachment and count_tokens-failure-survives tests
- sessions.test.ts: rewrote the two savings tests against the new
baseline_tokens + usage formula
Sending the pre-compression body to /v1/messages/count_tokens means
Anthropic sees the original text, which is exactly what pixelpipe is
built to prevent. Probe results were 'free' in dollars but not in
privacy; the whole point of imaging the slab is that Anthropic does
NOT receive the slab text.
Removed:
- countTokensUpstream + buildCountTokensBody in src/core/proxy.ts
- measurePromise wiring + finalize await
- baselineTokensMeasured / actualTokensMeasured on TransformInfo
- baseline_tokens_measured / actual_tokens_measured on TrackEvent
- measuredEvents / effectiveCostActualMeasured / *Baseline*
fields on DashboardState
- measured_events / saved_pct_measured / effective_cost_*_measured
/ saved_effective_tokens_measured on /proxy-stats payload
- HTML headline branch + 'show calculation' rows for measured path
- 2 dashboard-fit tests asserting the measurement-path behavior
Added regression test: proxy must NOT hit any upstream path other than
/v1/messages.
Real-savings path now: self-derive α from billed usage + own char
walker. Coming next.
count_tokens accepts only model/messages/system/tools/tool_choice/thinking/
mcp_servers. Forwarding the verbatim /v1/messages body included stream,
max_tokens, temperature, metadata etc, which the endpoint rejected with
400. Probe returned null silently → measured_events stuck at 0.
Strip to the accepted fields before posting. Now both probes succeed,
the dashboard headline can use saved_pct_measured = (Σ baseline −
Σ actual) / Σ baseline — exact, no regression, no priors.
Diagnostic logging kept on the probe path: failures surface in stderr
instead of disappearing.
Three changes addressing operator feedback after seeing the math panels:
1. Passthrough kill switch (the main ask). When upstream is unhealthy
or the proxy is misbehaving, the operator needs to disable
compression instantly without restart. Adds:
- DashboardState.compressionEnabled (in-memory, defaults true)
- POST /api/compression {enabled: bool}
- compression_enabled field on /proxy-stats
- Red banner "PASSTHROUGH MODE" + toggle button at top of dashboard
- node.ts transform fn returns {compress:false} when off, so the
proxy forwards /v1/messages unchanged to upstream
- In-memory only — restart resets to enabled. No CLI flag, no env
var. Runtime control via dashboard.
2. $ saved card now shows a range when one is available, mirroring
the % card. Previously the % was "0.3%–11.7%" but $ was just
"$0.2569" — visually inconsistent. New behavior: compute usdLo/usdHi
from saved_effective_tokens_{low,high} × input_per_mtok; show range
when spread ≥ 1¢, point otherwise. Both cards now feel like the
same headline.
3. Math-panel jargon replaced with plain English. The previous text
said "p10/p90 of per-sample α across fit ring" — incomprehensible
without statistics background. Now reads "low/high bounds from
recent traffic's chars-per-token spread" on both savings-tokens and
reduction-% panels.
Tests: 277/277. Typecheck + build clean.
Round-3 measurement on 2026-05-19 with 128-turn history showed the
feature loses money: 21K chars of text replaced with 140K tokens of
imagery, net -250% savings per request on this traffic shape.
The MARGINAL warning has been in the code comments since the feature
shipped - it was opt-in via --history / COMPRESS_HISTORY=1 originally
for exactly this reason. The recent flag-removal commit flipped it on
as default without re-verifying the savings claim against current
traffic.
The math in baselineCost() correctly reports negative savings when
imgTokensEst > txtReplaced, so the dashboard was honest about the
loss - the bug is shipping the feature on by default before measuring
a positive delta on representative traffic.
Re-enable per-deployment via env var when measurement confirms a
positive saved_pct on the specific shape.
Each headline card (tokens saved / $ saved / reduction %) gains a
collapsible "show calculation" panel that lays out the exact formula
plus the live numbers feeding it. Every digit on the headline can be
derived from what the panel shows — no estimation hidden behind a
black box.
Panels:
- tokens saved: actual_paid, baseline_est, saved formula
weights: input ×1.0, cc ×1.25, cr ×0.10, output ×5.0
- $ saved: formula = saved_tokens × input_per_mtok / 1e6,
exposes pricing_assumptions (rate ratios + source URL)
- reduction %:
- measured path (when count_tokens succeeded on ≥1 event):
baseline_tokens, actual_tokens, saved, saved_pct, n samples
labeled "ground truth, no estimation"
- estimated path (no measured events yet):
α-regression baseline, actual, saved, range from p10/p90
labeled "estimated · α-regression from cold-miss events"
- image-cost formula reference: tokens ≈ width × height / 750
Implementation:
- Native <details> elements, no JS framework. summary is a small
blue toggle that opens an inline formula block styled like a code
fence (dark background, monospaced).
- renderSavingsMath(s) is called once per /proxy-stats tick from the
existing fast-tick path. fmtRow builds key-value-note lines so the
three panels share a layout.
- pricing_assumptions block (input_per_mtok, output_multiplier,
cache_*_multiplier, source URL) already exists on /proxy-stats from
the earlier pricing-correction commit; this surfaces it inline.
No new endpoints, no new fields. The transparency comes entirely from
re-using existing /proxy-stats data and laying it out plainly.
Tests: 277/277. Typecheck + build clean.
The proxy used to expose ~20 behavior toggles (--no-compress,
--no-tools, --no-schemas, --no-reminders, --no-tool-results,
--history, --no-history, --history-keep-tail N, --history-min-prefix N,
--min-chars N, --min-reminder-chars N, --min-tool-result-chars N,
--cols N, --multi-col N, --no-track, --events-file PATH, --port N,
--upstream URL) plus matching env vars. Each one was a dead branch in
the common case and a configuration knob nobody had a principled
reason to pick a value for.
Replaced with: exactly one way to run the proxy. Every compression
mode on, every tuning parameter at its measured-best value, history
compression always active (was opt-in dead code). The only adjustable
surface is deployment concerns — where to listen, what to forward,
where to log — env-var only:
PORT default 47821
ANTHROPIC_UPSTREAM default https://api.anthropic.com
PIXELPIPE_LOG default ~/.pixelpipe/events.jsonl
CLI accepts only --help and --version. Anything else exits with
non-zero and a one-line error.
Notable behavior change:
- DEFAULTS.compressHistory flipped false → true. Variant-C history-
image compression now runs on every request (was opt-in via
--history / COMPRESS_HISTORY=1). The static-slab cache_control
placement is on the system image, not on the history-replacement
chain, so the published cache-topology risk does not apply.
Touched:
- src/node.ts: CliOpts → RuntimeConfig (3 fields). parseCli is now
~25 lines. printHelp reduced to env-var-only doc.
- src/core/transform.ts: DEFAULTS.compressHistory = true. Comment
updated.
- scripts/restart.sh: drops PROXY_ARGS array entirely. Port read
from PORT= env var. Unknown args rejected.
- tests/restart.test.sh: replaces flag-passthrough test with a
reject-unknown-args test. 4/4 pass.
- tests/history.test.ts: renames the "compressHistory:false (default)"
test to reflect the new always-on default; asserts the input-not-
mutated invariant remains.
- README.md: configuration table reduced to 3 env vars; restart
examples drop CLI flag passthrough.
Tests: 277/277. Typecheck + build clean. Restart script tests: 4/4.
The measurement code was previously gated behind a measureSavings flag
(--measure-savings CLI, PIXELPIPE_MEASURE_SAVINGS env var, optional
ProxyConfig field). That's a knob nobody asked for and a second
codepath whose only purpose is to leave the dashboard reporting
estimates instead of ground truth.
The flag is wrong for this codebase:
- Single codepath is the convention. The proxy either measures or
doesn't — there's no useful "sometimes" mode.
- count_tokens is free, runs in parallel with the main upstream
forward, and doesn't add user-visible latency.
- An opt-in default means the headline dashboard number stays as an
estimate forever in 99% of deployments — defeats the whole feature.
Removed:
- ProxyConfig.measureSavings field
- CliOpts.measureSavings field
- --no-measure-savings CLI handler
- PIXELPIPE_MEASURE_SAVINGS env var
- The if (config.measureSavings) wrapper in proxy.ts
- Help-text entry
- Stale comment references in tracker.ts, transform.ts, dashboard.ts
The proxy now unconditionally fires count_tokens on both bodies for
every /v1/messages request. Failure is silent — when count_tokens
returns null the event still emits without the measured fields and
the dashboard falls back to the regression estimate per existing
fallback logic.
Tests: 277/277. Typecheck + build clean. No behavioral test changes;
the existing measurement-path test exercises the synthetic event
shape directly, not the flag.
Replace estimation with measurement. Anthropic's /v1/messages/count_tokens
returns the exact tokenizer output for any request body, free of charge.
Calling it on the pre-transform body and the post-transform body gives
the EXACT delta in input tokens, with no α/β regression, no fallback
constants, no uncertainty band needed.
Opt-in (default off, since it adds two parallel HTTP roundtrips per
/v1/messages — count_tokens itself is free but the latency is real):
CLI flag: --measure-savings
Env var: PIXELPIPE_MEASURE_SAVINGS=1
When enabled the proxy fires both count_tokens calls in parallel with
the main upstream forward (no critical-path latency hit) and the event
is held for emit until both probes complete. Failures are silent — the
event still emits without the measured fields and the dashboard falls
back to the estimate.
Per-event new fields (events.jsonl):
baseline_tokens_measured exact pre-transform input tokens
actual_tokens_measured exact post-transform input tokens
Dashboard:
/proxy-stats gains: measured_events, effective_cost_actual_measured,
effective_cost_baseline_measured, saved_effective_tokens_measured,
saved_pct_measured. All null when no measured events exist; non-null
once ≥1 lands.
HTML headline switches to the measured number whenever measured_events
≥ 1. Subtitle reads "measured · count_tokens on N requests" to make
the source explicit. Falls back to the estimated range otherwise.
Measurement math:
delta_tokens = baseline_measured - actual_measured
bill_rate = (cc/total)·1.25 + (cr/total)·0.10 [same cache mix
as the actual call]
baseline_eff = actual_eff + delta_tokens · bill_rate
saved_pct_measured = (baseline_eff - actual_eff) / baseline_eff × 100
No α. No β. No FALLBACK_*. The numerator (delta_tokens) is the exact
output of Anthropic's tokenizer on both bodies; the multiplier is the
billed cache mix that already happened.
Tests: 275 → 277 (+2). Ground-truth path: 3 events with known token
deltas yield the expected saved_pct_measured. Fallback path: when no
event carries measurement, all saved_*_measured fields are null.
Typecheck + build clean.
Audit against published Anthropic docs (pricing + vision pages,
verified 2026-05-19 via WebFetch) found three constants in this file
that were either wrong or hand-picked without citation. Replace them
with documented values and add a PROVENANCE block so the lineage is
auditable.
Corrections:
1. saved_usd headline rate: $15/Mtok → $5/Mtok.
Opus 4.7 base input rate per docs is $5/Mtok, not $15. The old
value was Opus 3 pricing repurposed by the field name "_opus47".
The headline dollar number was therefore overstated by 3× on any
Opus 4.x deployment. Renamed field to saved_usd_estimated and
exposed pricing_assumptions{input_per_mtok, multipliers, source}
on /proxy-stats so the operator can verify what we multiplied by.
2. Per-image token cost: drop fixed OPUS_IMAGE_TOKEN_COST=2500 in
favor of the published formula tokens ≈ width × height / 750
(docs-vision). estImageTokens now prefers (in order): live β fit
→ published 1/750 → FALLBACK_IMAGE_TOKEN_COST (=1690, computed
from the formula at our default 808×1568 render). The old 2,500
constant traced to an in-house measurement that didn't reconcile
with the docs; pending a fresh per-renderer-config measurement
under post-walker-fix code.
3. FALLBACK_ALPHA point estimate: 0.25 → 0.33. Opus 4.7's tokenizer
note in [docs-pricing] explicitly states ~35% denser than older
models for the same text. CHARS_PER_TOKEN=4 (the older English
average) is therefore biased for Opus 4.7 traffic; 3 chars/tok
matches the documented behavior. The low/high brackets (0.15 /
0.50) span both tokenizer regimes and are left alone.
Added:
- PROVENANCE comment block at the top of the pricing/image-cost
section citing the three doc URLs and the verification date, with
each downstream constant footnoting back to one of the labels.
- pricing_assumptions object in /proxy-stats payload exposing every
multiplier and the source URL.
- Test pinning the pricing_assumptions shape and the
saved_usd_estimated = saved_tokens × input_per_mtok / 1e6 identity.
What this does NOT yet correct:
- Live α and per-renderer β still require fresh measurement post-
walker-fix. The fallback brackets keep saved_pct conservative
until the regression converges.
- 1-hour cache writes (2× input) are exposed in pricing_assumptions
but not yet distinguished from 5-minute writes in the cost math —
needs an upstream signal we don't currently capture per-event.
Tests: 274 → 275 (+1). Typecheck + build clean.
effectiveCost() previously summed only input + cache_create·1.25 +
cache_read·0.10, leaving output tokens out of the cost total. The
headline saved_pct was therefore "% saved on the input-side bill"
rather than "% saved on the full upstream bill." Output is small in
absolute terms on typical Claude Code traffic (≈0.4% of cost in
the sample events.jsonl), but the framing is what matters.
Changes:
- effectiveCost gains an outputTokens parameter and sums output ×
OUTPUT_TOKEN_RATE (=5.0, the Opus/Sonnet output:input rate ratio).
- Totals fields renamed effectiveInput* → effectiveCost* to match
the new full-bill semantics. Same on the JSON wire:
effective_input_* → effective_cost_*.
- update() now also passes the event's output_tokens into
effectiveCost so both actual and baseline grow consistently.
- replay() effective_actual computation includes output_tokens too,
so historical rows align with the live formula.
- Headline subtitles updated to "share of total bill saved" and
"effective tokens (full bill)" so the unit is self-documenting.
Math invariant: output is identical in actual and baseline (the model
produces the same response regardless of prompt compression), so it
cancels in the numerator saved = baseline - actual but inflates the
denominator. saved_pct therefore moves SMALLER and more conservative
when output is non-trivial — honest whole-bill terms.
New test pins the invariant: two identical input-side runs with 100×
different output_tokens give the SAME saved_effective_tokens but
strictly different saved_pct, with the higher-output run lower.
Tests: 273 → 274 (+1). Typecheck + build clean.
The break-even gate (isCompressionProfitable) used a hard-coded
CHARS_PER_TOKEN=4 (Anthropic's English avg). Live Claude Code traffic
has α≈0.5-0.9 (1.1-2 chars/tok) — JSON-dense tool docs, structured
CLAUDE.md slabs, etc. Off-by ~3.5× on the text side at α=0.88.
Production trace (events.jsonl, 2026-05-19): 200+ requests with system
slabs 113K-170K chars hit reason="not_profitable" because the gate's
textTokensEq = len/4 was a fraction of the row-aware image-token cost.
Example event: slab=169_632 chars, cache_create_tokens=148_891. Gate's
textEq=42K rejected vs imgCost=127K. With live α (1.14 ch/tok),
textEq=149K → ACCEPT. We were leaving real cache_create money on the
table on every one of those calls.
- TransformOptions gains charsPerToken?: number. Default 4 (back-compat).
- isCompressionProfitable() takes charsPerToken as a 5th optional param;
defensive clamp on ≤0 / NaN / Infinity falls back to CHARS_PER_TOKEN.
- All 4 internal gate call sites (static slab + reminder + tool_result
string + tool_result array) thread o.charsPerToken through.
- ProxyConfig.transform now also accepts a function: () => TransformOptions.
Resolved per-request inside createProxy so the host can inject dynamic
values. Static-object form unchanged for Workers / tests.
- node.ts wraps the static transform options in a closure that asks
dashboardState.fitCosts() each request and passes the live
chars_per_token. Fit-null falls through to the baked-in default — no
chicken-and-egg deadlock since normal-sized slabs still feed the fit
ring even before the huge-slab cases get unblocked.
- 4 new gate tests: stale-default rejects the 170K newline-heavy slab;
live α=1.14 flips the same slab to profitable; defensive clamp on
bogus charsPerToken; end-to-end transformRequest passes through.
Tests: 269 → 273 (+4). Typecheck + build clean.
The sessions table previously only supported per-row deletion via the
"del" button — no way to clear many at once. Add an industry-standard
multi-select pattern.
Pattern follows Gmail / GitHub / Linear / Vercel:
- Leftmost cell on each row is a checkbox; data-id binds to session id.
- Header checkbox is tri-state (empty / indeterminate / all-on-page)
driven by a single source of truth (selectedSessionIds: Set<string>).
- Shift-click on a row checkbox does range-select from the
last-clicked anchor to the current row, applying the new checked
state to every row in between (macOS Finder / Gmail convention).
- Contextual action bar slides in above the table when ≥1 row is
selected, showing "N selected · Delete selected · Clear (Esc)".
- Confirm modal shows count + first 3 session id prefixes + "and N
more" for sanity-check before destructive action.
- Esc key clears the selection (no-op when an input is focused).
- Selection is keyed on session id and reapplied after each diff
render — innerHTML rewrites no longer clobber checked state.
- Vanished sessions auto-drop from selection.
Backend:
- PruneOptions gains sessionIds?: string[] alongside the existing
single sessionId. Unknown ids are silently skipped (race-safe with
concurrent prunes). Composes with sessionId if both are set.
- handlePrune plumbs sessionIds through; the bulk-delete action calls
/api/sessions/prune once with all ids.
- 3 new selectSessionsToRemove tests: bulk select, unknown-id skip,
sessionId+sessionIds composition.
Tests: 266 → 269 (+3). Typecheck + build clean.
The headline used to publish a precise saved_pct (e.g. "68.2%") even
when α was thin / fallback. Switch to a range whenever uncertainty is
wide enough to matter — point estimate only when the spread is tight.
- fitCosts() returns alpha_low / alpha_high (and chars_per_token bounds)
from p10/p90 of per-sample residual α = (tokens - β·pixels) / chars
across the fit-sample ring. For n≤3 uses min/max; for larger n uses
10th/90th percentile. Order-clamped against the central point so a
noisy sample can't put low > point or high < point.
- baselineCost() gains an optional alphaOverride knob that varies ONLY
the chars-per-token side while keeping the image-cost regime stable
(β·pixels when fit is present, 2500/img fallback otherwise). Without
this knob, an earlier draft mixed regimes and produced
saved_pct_low > saved_pct — silently incoherent.
- Totals tracks three parallel baselines: effectiveInputBaselineEst
(point), …Low, …High. Every event updates all three.
- /proxy-stats exposes saved_pct, saved_pct_low, saved_pct_high plus
the matching effective_input_baseline_est_{low,high} and
saved_effective_tokens_{low,high}. Bounds clamped ≥ 0.
- HTML headline collapses to point estimate when (high − low) < 5 pp;
otherwise renders "lo–hi%" as the headline with point as subtitle.
- When fit is null (n<3), wide fallback brackets (0.15 / 0.25 / 0.50)
produce a deliberately wide range so the headline shows a calibrating
state instead of a fake-precise point estimate.
- 5 new tests: alpha_low ≤ alpha ≤ alpha_high invariant, chars/tok
bound inversion, serveStats bound monotonicity, null-fit range shape,
and warmup-baseline ordering.
Tests: 261 → 266 (+5). Typecheck + build clean.
The walker was missing three big text-bearing paths the upstream
tokenizer sees: top-level `tools[]` (name + description + JSON schema),
`tool_use` blocks (name + serialized input), and `thinking` blocks.
Under-counting these inflates α in `tokens ≈ α·textChars + β·pixels`,
which biases the dashboard's `saved_pct` HIGH — the honesty cost
HANDOFF item #1 called out.
- Walker now sums: system + tools[].{name,description,input_schema} +
per-block text/tool_use.input/tool_result.{tool_use_id,content}/
thinking. JSON.stringify via safeStringifyLen() for object fields
so the count matches what the API actually serializes.
- Lift the walker call out of the compression-success path: also run
it on the below_min_chars and not_profitable early returns. Without
this, α was sampled only from "big enough to compress" requests.
- New `outgoingTextChars walker (denominator honesty)` describe block
in render.test.ts: 7 tests covering baseline, tools[], tool_use,
tool_result (string + array forms with image skip), thinking,
unknown-block skipping, and a JSON.stringify(req).length upper bound
asserting walker ≥ 50% of envelope on a content-dense request.
- Confirms walker stays strictly below the JSON envelope (structural
keys/braces still excluded — that's intentional).
Tests: 254 → 261 (+7). Typecheck + build clean.
Live diagnosis: real steady-state single-session traffic has 1-2% text
CV turn-to-turn (verified 2026-05-19: six consecutive turns at 132,738
/ 133,021 / 133,610 / 134,105 / 134,718 / 138,097 chars = 1.33% CV).
The previous 2% gate I'd set was still too tight — the constrained fit
never activated, dashboard stayed on stale-constants regime indefinitely.
The CV gate was over-protective. With β pinned, α = Σ(r·x)/Σ(x²) is
mathematically well-defined and numerically stable at any cvX > 0. The
gate only matters for splitting two collinear parameters (joint mode).
For constrained mode I'm replacing the gate with a confidence display:
- MIN_CV_CONSTRAINED = 0.001 (0.1% — purely defensive against
byte-identical sample collapse)
- text_cv + pixels_cv added to cost_fit JSON response
- badge surfaces 'text CV X.X%' so operator sees confidence directly
- constrained-mode badge color tracks CV:
blue = robust (CV ≥ 5%)
amber = thin but measured (1% ≤ CV < 5%)
gray = mean-ratio estimator (CV < 1%)
This is the right shape: don't gate, show the number. The operator can
judge whether '38% saved at text CV 1.3%' is trustworthy enough.
Tests:
+ new: 6-sample 1.33% CV fixture activates constrained mode + reports
cv fields
+ replaced: 'below 2% floor rejects' → 'byte-identical (cv=0) rejects'
unchanged: joint mode + collinear-pixels constrained mode
Live-traffic diagnosis: typical single-session warm-cache load has the
same cached image across turns (pixel CV = 0%) AND text variance of
3-4% (verified 2026-05-19: 117k/127k/128k chars across three turns =
3.84% CV). Both columns failed the original 5%-everywhere gate, so the
constrained fallback I just shipped never actually activated — the
dashboard fell straight through to wandering stale constants.
The 5% gate exists to stop joint OLS from splitting two indistinguishable
parameters. In constrained mode β is pinned (no split to make), so we're
fitting one parameter through the origin — 1-D OLS is numerically stable
at much lower variance. 2% is conservative; even 1% would converge.
New thresholds:
MIN_CV_JOINT = 0.05 (unchanged — both columns must vary)
MIN_CV_CONSTRAINED = 0.02 (NEW — text only, β-pinned through origin)
Real-world impact: live single-session traffic now activates constrained
mode after 3 compressed turns instead of waiting for cross-session
variance that may never arrive. Dashboard label flips from amber 'stale
constants' to blue 'constrained · α-only' — operator sees a measured α
within ~30 seconds of starting a fresh proxy.
Tests:
+ new: live-traffic-shape sample activates constrained mode (3.84% CV)
+ new: below-2% floor still rejects (no garbage 1-D fits)
unchanged: 2-D collinear cases still null, joint-fit at high CV still
returns mode='joint'
Paint a 1-pixel-wide mid-gray (191/255) vertical line in the center of
each gutter at multicol >= 2. Same 'visible whitespace' principle as the
U+2192 tab arrow — surfaces structure that pure whitespace alone leaves
ambiguous, at near-zero pixel-token cost (DEFLATE collapses identical
mid-gray runs to ~5 bytes per gutter).
Why now: the user noted gutters relied on whitespace alone, which is
exactly the OCR-reordering risk HANDOFF flagged for multi-col 2 (lines
322-326). An explicit boundary cue lets us push wider multicol or wider
cols in future without spooking the vision encoder into reading
left-to-right across columns.
Tests:
- byte-determinism (existing test still passes)
- inflate IDAT + assert mid-gray pixels at expected divider X
- numCols=1 path stays byte-identical to renderTextToPngs (deterministic
cache_control story for single-col deployments)
Cost: ink=64 pre-invert → 191 post-invert. Doesn't compete with glyph
ink (which lands at 0). At β=0.001776, the divider's ~1568 pixels cost
~3 tokens total per multi-col image. Trivial vs the OCR-clarity win.
When the live OLS can't identify β separately (pixels column collinear
because a single cached image dominates the 50-sample ring), fall back to
α-only fit with β pinned to Anthropic's published rate. Returns
mode='joint' | 'constrained' so the HTML can tell the operator which
regime the headline saved_pct came from.
Three-state badge under the reduction card:
- green 'empirical · joint OLS (n≥10)' = both rates measured, confident
- amber 'empirical · joint OLS (n<10, tentative)' = both measured, thin
- blue 'constrained · α-only (β pinned 1/750)' = α measured, β=Anthropic
- amber 'stale constants (no fit yet)' = pre-fit, treat skeptically
Closes the 'wandering ±15pp until the fit fires' gap from HANDOFF: even
under single-session traffic the dashboard now shows a measured α rather
than oscillating between joint-OLS-degenerate and stale-constants.
# The user-visible bug
Showed wandering saved_pct from oscillating OLS:
n=4: α=0.509, β=0.00387, saved_pct=20.1%
n=8: α=0.879, β=0.00104, saved_pct=25.0% (same session, 4 more samples)
# The root cause
A single Claude Code session sends warm hits with the SAME cached image
on every turn → `pixels` column has ZERO variance across samples. The
regression `tokens ≈ α·text + β·pixels` only identifies α and β
SEPARATELY when both columns vary. With pixels constant the model
degenerates to `tokens ≈ α·text + c` where c = β·p̄ — OLS distributes
the per-sample cost between α and β however the residual noise pushes,
and the split flips wildly as samples accumulate.
The det != 0 numerical check missed this because text has tiny non-zero
variance, so the matrix isn't EXACTLY singular — just near-singular,
which is enough for the headline to wander by ±15 pp.
# The fix
Add a coefficient-of-variation guard. Both columns must have CV (stdev
÷ mean) > 5% before fitCosts() returns non-null. Below that, the fit
is statistically meaningless and we fall back to the stale-constants
baseline. Better to show a stable known-wrong number than an unstable
unidentifiable one.
This is the OPPOSITE direction from the earlier 'cache_read===0' gate
relaxation (which was the right call — warm hits DO contribute to the
sample ring), but at a different layer:
- Gate at sample ingestion: liberal (any compressed request with
full usage triple)
- Gate at fit ACTIVATION: strict (require non-collinear samples)
# Known remaining biases (HANDOFF documents)
1. `outgoing_text_chars` undercounts — doesn't include `req.tools`
text. Measured slope α≈1.1 across collinear-pixels samples implies
~0.9 chars/token, vs Opus 4.7 reality 2.7-3.6. Probably ~half the
tokenized chars are missing from our walker.
2. Even with #1 fixed, β identification needs cross-session variance
(samples with different cached images). Either wait for cache TTL
expiry, drive multi-session traffic, or add a constrained-fit
fallback that pins β = 1/750 (Anthropic's published rate) and
solves α only.
# Tests
tests/dashboard-fit.test.ts: replaced the warm-hit-with-constant-pixels
test (which passed only because the old code had no CV guard) with two
new tests:
- returns null when pixels column is constant (single-session case)
- returns null when text_chars is constant (mirror case)
- activates the fit when both columns vary > 5% (cross-session-style
samples reconstruct α≈0.286, β≈1.5e-3 to within ~10%)
248 tests pass (up from 246).
# Milestone status
The dashboard is now HONEST (refuses to claim numbers it can't
measure) but NOT yet CALIBRATED (still on stale constants until cross-
session samples land). HANDOFF.md captures the path to calibration.
The earlier gate (`cache_read === 0 && cache_create > 1000`) was wrong.
Anthropic's tokenizer is deterministic on the request bytes — cache state
changes BILLING (cache_read is discounted), not token count. The total
tokenization of the request body is always:
total_tokens = input + cache_create + cache_read
…regardless of how it was split across cached/uncached portions. So warm
hits carry the same body-token information as cold misses; the regression
`tokens ≈ α·text_chars + β·pixels` is solvable from either.
The old gate locked the fit out of all normal traffic: warm hits ARE the
steady state once a Claude Code session has run a few turns. The user
correctly noticed that cost_fit stayed null forever even with the proxy
processing real requests, because every request after the first hits the
prompt cache.
# The fix
- Remove the `cache_read === 0` requirement.
- LHS becomes `input + cache_create + cache_read` (was `input + cache_create`).
- Keep the 1000-token floor (filters trivial no-system requests where
tokenizer overhead dominates).
- Rename `coldMisses` / `ColdMissSample` / `COLD_MISS_CAP` → `fitSamples`
/ `FitSample` / `FIT_SAMPLE_CAP` since the samples are no longer cold-
miss-gated.
# Collinearity guard already handles the single-session case
The OLS no-intercept solver uses det = sxx·syy − sxy². With pixels held
constant at p̄ (same cached image across a session's warm hits) but
text_chars varying turn-to-turn, det = p̄²·(n·Σx² − (Σx)²) which is
positive by Cauchy-Schwarz whenever text varies. So a single Claude Code
session can populate the fit on its own — no need to spawn N sessions.
# Tests
New tests/dashboard-fit.test.ts (5 cases) locks in:
- null below 3 samples
- warm cache hits DO seed the ring (the regression we're guarding)
- LHS = input + cache_create + cache_read (cold vs warm with identical
body lands on the same regression line)
- 1000-token floor still filters trivial no-system traffic
- compressed=false passthrough requests don't contaminate the fit
246 tests pass (up from 241), typecheck + build clean.
The `placement === 'system'` branch attached image blocks to the `system`
field, which Anthropic 400s with `system.N.type: Input should be 'text'`.
The existing CLI help text already warned about this. No tests exercised
the dead branch (grep for `placement.*'system'` finds zero call sites).
`compressSystem` was only consulted inside that same dead branch, so it
went out with it. The other granular `compress{Tools,Schemas,Reminders,
ToolResults}` flags + the `compress` master switch stay — they're useful
debugging toggles with active call sites.
Touched:
- transform.ts: drop both fields from TransformOptions + DEFAULTS;
collapse the if/else around system-rewriting into a bare block (no
re-indentation noise, no behavior change).
- node.ts: drop --placement CLI flag, PLACEMENT env, help text entries,
startup log column, and the TransformOptions wire-up.
- worker.ts: drop PLACEMENT from Env + transform options.
- render.test.ts: refresh three comments that referenced the now-gone
"default placement is 'user'" language; the test bodies were already
correct and didn't reference the removed knobs.
20-line net reduction, 241 tests pass, typecheck + build clean.
`DashboardState.fitCosts()` runs OLS over a rolling 50-event ring of
cold-miss (image_pixels, outgoing_text_chars, cache_create_tokens)
samples. `baselineCost()` and `estImageTokens()` consult the fit on
every event:
- `cost_fit != null` (N ≥ 3 samples) → headline saved_pct switches to
the live empirical rates: txtReplaced = compressedChars × α,
imgTokens = imagePixels × β.
- `cost_fit == null` → falls back to the stale constants
(OPUS_IMAGE_TOKEN_COST = 2500, CHARS_PER_TOKEN = 4) — same behavior
as before, so nothing breaks while data accumulates.
# Why a rolling ring, not seeded from history
Old events may carry the new instrumented fields but come from a
different renderer config (cell size, multiCol, atlas profile) or a
different model (Opus 4.6 vs 4.7 tokenizer). Mixing them in gives a
misleading α/β that lags reality. Each restart starts the fit empty
and refills from live traffic, so flipping `multiCol`, swapping
fonts, or upgrading the model auto-flushes stale samples.
# Endpoint
`/proxy-stats` JSON payload now carries:
"cost_fit": {
"alpha": 0.3, "beta": 0.0053,
"chars_per_token": 3.3, "pixels_per_token": 189,
"single_col_tokens_per_img": 4180,
"multicol2_tokens_per_img": 8500,
"n": 12
}
null until ≥3 cold-miss samples land. Guards against degenerate fits
(collinear data, negative rates) by returning null rather than feeding
the dashboard noise.
# Math
Solves `tokens ≈ α · text_chars + β · pixels` via 2×2 normal
equations on the centered design matrix. Rounds α to 4 sig figs, keeps
β at 6 sig figs (β is tiny ~5e-3 so it needs the resolution).
# Open
Once N stabilizes at ~10+, re-grade the hardcoded
`OPUS_IMAGE_TOKEN_COST` and `CHARS_PER_TOKEN` constants to match the
empirical numbers. That just makes the fall-back path agree with
reality, doesn't change live behavior. HANDOFF.md captures the
verification recipe and priority queue for the next session.
Two pieces of work landing together because they're interleaved in
transform.ts and need to ship at the same time for the dashboard
fit-wiring in the next commit to work.
# R2 multi-column rendering (default `multiCol=2`)
Single-col packing wastes ~68% of Anthropic's 1568×1568 image budget
on real Claude Code tool docs (~38 chars/row). The row-aware break-
even gate correctly rejects compression on those slabs — proxy ends
up doing nothing on the heaviest requests. Multi-col packs N text
columns side-by-side per image so each image covers `N×LINES_PER_IMAGE`
wrapped lines, dropping image count by ~N and crossing break-even.
- render.ts: new `renderTextToPngsMultiCol(text, cols, numCols)` with
column-major layout (col 0 fills top-to-bottom, then col 1, …).
Plus `multiColWidth` / `maxFittingCols` helpers. CLI auto-clamps
numCols so the canvas can never exceed the 1568 px width cap.
- transform.ts: `multiCol` option threaded through; gate, image-count
estimator, paging budget, and all render paths take `numCols`.
Intro text now describes column reading order when numCols>1 so
the OCR pass doesn't read row-wise across columns.
- Host wiring: node.ts (--multi-col / MULTI_COL=N, default 2),
worker.ts (MULTI_COL env, default 2). Startup log shows multi_col=N.
- numCols=1 stays a byte-identical passthrough to renderTextToPngs
so flipping back can't regress the cache_control hit rate.
# Empirical-cost instrumentation
The dashboard's `OPUS_IMAGE_TOKEN_COST = 2500` and `CHARS_PER_TOKEN = 4`
are stale pre-Opus-4.7 constants. Fact-checked: 4.7 changed the TEXT
tokenizer only (~1.20× inflation on English, ~1.36× on TypeScript);
image cost is unchanged for same-size images. Direction of error in
the dashboard headline: it UNDERSTATES text savings.
Ditched the "guess from one cold-miss" approach in favor of real
instrumentation that pins both axes at once:
- TransformInfo.imagePixels — Σ width×height across every image
this request (static slab + reminders + tool_results + history
collapse). Threaded through textToImageBlocks, collapseHistory.
- TransformInfo.outgoingTextChars — every char of TEXT remaining
in the transformed outgoing body, computed in one walk before
serialization (countOutgoingTextChars).
- Tracker event schema picks up image_pixels + outgoing_text_chars.
- history.ts: HistoryCollapseInfo.collapsedImagePixels accumulates.
The regression `tokens ≈ α·outgoing_text_chars + β·image_pixels`
over N cold-miss events solves for empirical chars/token (α) and
pixels/token (β) under the live model — no assumed constants.
# Tests
- 8 new render tests covering the multi-col passthrough contract
(numCols=1 byte-identical to renderTextToPngs), canvas dimensions,
image-count halving, determinism, charsRendered sum, estimator
parity, CLI clamp, and CJK wide-glyph wrap.
- paging.test.ts "counts multiple tool_results" threshold loosened
from >770k to >700k to cover both single-col and multi-col budgets
at the same maxImagesPerToolResult cap.
- All 241 tests pass, typecheck + build clean.
renderToolDoc pretty-printed every input_schema with 2-space indent,
putting each property on its own line. At cols=100 this wasted ~70% of
horizontal space — schemas rendered to many short rows instead of a few
dense ones.
Live measurement on 2026-05-19: 150 KB of pretty-printed tool docs in a
177 KB combined slab → 31 static-slab images per request at 40% fill.
With the new row-aware break-even gate (f0176c1), those rejected as
not_profitable — no savings on the user's typical workload.
Synthetic measurement on 30 realistic tool schemas:
- pretty: 320,843 chars / 14,074 lines → 100 images
- compact: 124,720 chars / 179 lines → 10 images
- 2.57× char shrink, 78× row shrink, 10× image cost reduction
Compact JSON is unambiguous: descriptions are already stripped under
compressSchemas, so only structural keys (type/properties/required/
enum/items) remain — those read fluently on one wrapped row. Frontier
models handle compact JSON natively; it's the default wire format for
tool_use blocks anyway.
Test pins the density gain on a representative schema.
teamchong