The system slab's break-even gate was estimating image cost at the full
configured cols width while the renderer was shrinking the canvas to the
longest wrapped line. Gate and renderer now agree: shrink once up front,
pass the shrunk cols to both evalCompressionProfitability and
renderTextToPngsMulticol.
Two fixes to the image-cost prediction that gates compression:
1. **Content-aware image cost.** The old gate used a fixed
`TOKENS_PER_IMAGE_SINGLE_COL = 2500` constant per image, modelling
every tool_result block as a full 1568px canvas. The renderer
actually produces images sized to the wrapped line count, so a
5,000-char tool_result becomes a ~80px-tall image costing ~54 tokens,
not 2,500. Result: the break-even for small blocks was over-estimated
by ~50x and the gate rejected anything below ~10k chars even though
image-form was cheaper.
Replace `estImages × effectiveTokensPerImage(numCols)` with a single
content-aware `imageTokensCost(text, cols, numCols, imageCountCap)`
that mirrors the renderer's height math:
`tokens = (rows + 1) × overhead + fullImageRows × perRow`, derived
from the cold-miss anchor and applied row-proportionally for partial
final images. Calibrated against N=33 production cold-miss
measurements (median multiplier 1.05, mean 1.32 — high-image case is
~1.05x of the documented `width × height / 750` rate).
2. **Width-shrinking on per-block paths.** For per-block call sites
(tool_result, reminder) the renderer now narrows the canvas width to
the longest wrapped line via `shrinkColsToContent`. Multi-col
layouts already pack tightly; this captures the single-col savings
that were previously paying for whitespace. The gate threads
`shrinkWidth` through to `evalCompressionProfitability` /
`isCompressionProfitable` / `isCompressionProfitableAmortized` so
the prediction matches what the renderer will actually produce.
System-slab and history-collapse paths pass `shrinkWidth=false` —
they fill the configured canvas deliberately.
Knobs removed: `MinToolResultChars` and `MinReminderChars` now
default to 0 (let the math decide). The TransformOptions fields are kept
for env-override observability but no longer set a floor in production.
`isCompressionProfitable` / `evalCompressionProfitability` /
`isCompressionProfitableAmortized` are now string-only at the public
API; the numeric-length fallback is gone (it under-counted images and
silently leaked net-losing compressions through).
Tests: gate-flap and history.test updated for the new physics, but 24
tests in render.test.ts still assert the OLD over-estimation as truth
(e.g. "5,000 chars is not profitable", "7000 chars stays as text").
They need to be deleted or have their expected verdicts inverted — left
for the next session to finish the rewrite cleanly. Typecheck passes.
Measured impact: the production tail's $1.43 flap is gone and per-block
tool_result/reminder content is now genuinely compressed when it would
save tokens, not just when it crosses the (wrong) 10k-char threshold.
Add per-path actual-billed cost counters to the dashboard alongside the
existing counterfactual baseline. The compressed-vs-passthrough split
sums what each code path actually cost on real traffic, partitioned by
`info.compressed`, with no probe gating or counterfactuals. Surfaces
sample counts in the UI so operators can judge sufficiency rather than
having the dashboard auto-claim significance.
Headline math also switched to share-of-all-paid-spend (counterfactual
denominator bounded at 100%) so the slice number never lies when
pixelpipe didn't run on every request. Regenerated dashboard-bundle.
The previous patch divided savings by all-rows ACTUAL spend, which is
not bounded — pixelpipe shrinks the denominator, so a single big
cold-miss compressed request showed '280% saved'. That's nonsense:
you can't save more than you would have paid.
Correct denominator is what the user WOULD have paid with no pixelpipe:
measured rows contribute their cache-aware baseline, unmeasured/probe-
failed/passthrough rows contribute their actual (pixelpipe either
didn't run there, or we can't measure the counterfactual, so actual ≈
baseline). Bounded at 100% by construction.
- Totals: rename allActualInputWeighted role to diagnostic only, add
allBaselineEquivalentWeighted as the headline denominator input.
- update(): when haveBaseline, add baselineInputEff to the counterfactual
pool; otherwise add actualInputEff. Output × 5 unchanged on both sides.
- serveStats(): pctAllSpend = saved / (allBaselineEquiv + allOutput).
- types.ts + StatsHeader.svelte: surface all_baseline_equivalent_weighted,
rewrite the explainer math.
Single-request smoke (the case that surfaced the bug): saved=397k
input tokens, baseline_equivalent≈400k, output_equiv tiny → ratio ≈
99% saved, which is honest (one cold-miss compressed turn on an
otherwise empty session).
The 'share of total bill saved' headline divided savings by
baseline_input_weighted, which only accumulated on rows where the
count_tokens probe succeeded AND the request had upstream usage. Every
passthrough turn, every probe-failed turn, and every flap-polluted
untransformed turn was excluded from the denominator. That's a
cherry-pick: it answers 'did pixelpipe help on the rows where it ran'
instead of 'did pixelpipe move my real bill'.
Track allActualInputWeighted / allOutputWeighted ungated on probe
success (any request with a usage block contributes), expose
saved_pct_of_all_spend on /proxy-stats, and rewrite the headline card
to use it. Numerator is still measured-rows-only because that's the
only place we have a counterfactual baseline; you can't fabricate
savings for rows where the probe failed. But the denominator now
matches the user's actual bill.
The legacy saved_pct_of_total_bill stays on the wire for back-compat
(marked deprecated in types.ts). The headline card no longer reads it.
Card flips color: positive % stays green, negative % goes red, because
flap-driven net losses are real and should be visible.
Production data (ocproxy claude-opus-4-7, 2026-05-23 09:19–09:21) showed
three-turn sessions still flapping AND paying cache_create on every
turn despite the symmetric burn shipped in 0cfe08a:
09:19 APPLIED cc=156k cr=6.7k $0.99 cold start
09:20 declined cc=181k cr=0 $1.23 FLIPPED to text
09:21 APPLIED cc=156k cr=6.7k $1.01 FLIPPED back to image
The slab gate's gateEval said image was profitable by 154k margin at
the flip turn, but applied=0 — the slab gate had decided to stay in
image mode, then the history-collapse gate (which got hardcoded 0 for
priorWarmTokens AND priorWarmImageTokens) flipped the verdict.
Wire o.priorWarmTokens + o.priorWarmImageTokens through to both
history-collapse call sites so the symmetric anti-flapping burn
applies to every gate that can change the session's bill shape, not
just the slab. Foundational follow-through on the same fix.
Production data (ocproxy Opus 4.7, 2026-05-22 23:15–23:44) showed the
slab break-even gate ping-ponging between text and image modes within a
single session, paying cache_create on whichever side it just flipped
away from. Two flips in 30 minutes cost ~$0.71 + ~$0.40 of avoidable
cache_create on a single conversation.
Root cause: priorWarmTokens was asymmetric. It penalised compressing
when the un-rewritten text path was warm, but did not penalise
decompressing when the rewritten image path was warm. Per-turn cost
favored flipping; the flip forced cache_create on the new side; next
turn flipped back.
This change adds the symmetric counterpart:
text→image flip: warm text invalidated → burn applied to IMAGE side
(existing behavior, priorWarmTokens)
image→text flip: warm image invalidated → burn applied to TEXT side
(NEW, priorWarmImageTokens)
Once a session commits to a mode, staying in that mode is cheaper than
flipping by the burn cost. Mode changes only happen when the per-turn
delta exceeds the burn, which is exactly when the flip is actually
worth the cache_create. Foundational fix — no thresholds.
Observability:
- New exported `evalCompressionProfitability()` decomposes the slab
gate's evaluation into imageTokens, textTokens, burnImageSide,
burnTextSide, and verdict. Same constants and helper as the gate
itself, no drift risk.
- New `TransformInfo.gateEval` field stamps that decomposition into
every applied or not_profitable event so hosts can persist the verdict
margin and prove flap-prevention efficacy from telemetry.
Back-compat:
- New params default to 0; single-arg callers behave identically.
- 315 existing tests unchanged; +7 new tests pin the new semantics
(back-compat, asymmetric ⇒ symmetric, both-sides-warm cancellation,
eval verdict matches gate verdict, burn term math).
The reflow-inimage finding (commit bb9c231) made 5×8 viable, so
DEFAULT_CELL_W_BONUS / DEFAULT_CELL_H_BONUS reverted to 0 — but many
comments and the README still said 7×10 was production.
Touched:
- README.md — eval table, density bullet, sweep narrative,
cell-pitch section, measurement header,
Limitations section
- src/core/render.ts — RenderStyle docstring + cell-bonus docstrings
+ multicol intro comment
- src/core/transform.ts — TransformOptions.reflow docstring, DEFAULTS
comments (minReminderChars floor rationale,
maxImagesPerToolResult cap, reflow gate
rationale, anchor scaling formula, LINES_PER_IMAGE
worked example)
No code behavior changed — every constant still resolves through CELL_W /
CELL_H, and the eval harness still overrides the cell via cellWBonus /
cellHBonus when comparing variants. The 5×8 production path is what was
already shipped on 81550f6 (reflow + grayscale + L1/L2 harness) and
bb9c231 (reflow-inimage).
Pack text into a continuous sentinel-delimited stream (↵ = U+21B5) so
wrapLines fills every row to `cols` instead of leaving dead right-margin.
Adds reflow/dereflow, renderTextToPngsReflow{,MultiCol} variants, a full
test suite, and an A/B eval harness with L1 OCR + L2 session results.
Replace the hand-rolled dashboard with a Svelte component tree (App +
Sessions / StatsTable / LatestPng / RecentRequests / StatsHeader /
Cleanup / CompressionToggle / ToastTray), move the API layer into
lib/, and split state into stores/.
Add a dedicated dashboard build (`build:dashboard-ui` + a separate
tsconfig.dashboard.json) so dashboard sources typecheck independently
of the proxy core, which previously broke the root `tsc` build.
Snap the collapse cutoff to a `collapseChunk`-sized grid (default 50) so
the rendered history image stays byte-identical for a full window of
turns. That lets Anthropic's prompt cache `cache_read` the history
prefix (0.1x) instead of re-billing `cache_create` (1.25x) every turn.
The cutoff floors at `minCollapsePrefix` (clamped to the raw cutoff) so
short conversations still collapse without the boundary drifting.
Also log `history_image_sha8` per request -- a sha8 of the concatenated
history-image base64. An unchanged hash across consecutive collapsed
events is ground-truth proof the prompt cache is being hit; a hash that
moves every turn signals the cache-key drift regression is back.
Production data (ocproxy 7d, Opus 4.7) shows pixelpipe HURTS short
conversations and helps long ones:
input.count <10 : +$0.17 (HURTS) over 14 rows
input.count 10-19: +$0.04 (HURTS) over 22 rows
input.count 20-39: -$0.56 (helps) over 29 rows
input.count 40+ : -$3.21 (helps) over 237 rows
Diagnosis: warm-cache share split.
bucket actual_warm_share baseline_warm_share
<10 60% 100%
10-19 84% 97%
20-39 88% 96%
40+ 92% 58%
For short conversations the un-rewritten path is already nearly fully
warm-cached. Rewriting the prefix bytes (slab compression / image
substitution / history collapse) gives the new prefix a different
cache key, so Anthropic charges cache_create (1.25×) on the new
prefix's first turn — destroying the prior warm cache. The
conversation often ends before the new cache amortizes.
Fix: add a 'priorWarmTokens' option to TransformOptions. When >0, the
break-even gate penalises the image side by
burn = priorWarmTokens × (CACHE_CREATE_RATE − CACHE_READ_RATE)
Per-turn gate eats the full burn; amortized gate spreads textLifetime
across N turns so longer horizons re-accept once savings outweigh burn.
Default 0 preserves byte-identical behavior for callers that don't
plumb it through (cold-start safe).
Tests: 3 new (NaN clamp, per-turn flip, horizon flip). 302 total pass.
count_tokens validates structural pairing and rejects truncated prefix
bodies that contain tool_use blocks whose tool_result was in the
dropped tail, with: 'messages.<N>: tool_use ids were found without
tool_result blocks'. That's been firing on ~15% of Opus 4.7 rows
(75/503 last 7d) and excluding them from baseline_probe_status='ok'.
Append a synthetic user message with minimal tool_result blocks for
any orphan ids. Adds ~5-10 tokens per orphan (within ~1% of truth)
and converts those rows from 'partial' to 'ok' so they contribute
to honest savings rollups.
Source: ocproxy daemon.log under OCPROXY_PIXELPIPE_DEBUG=1.
N=391 Opus 4.7 measured rows (last 7d) show real chars-per-token = 1.91
(avg_outgoing_text_chars=231,925 / avg_real_input_tokens=115,893). The
older 2.5 constant was calibrated on Opus 4.5/4.6 and over-estimated cpt
by ~30%, under-counting text-token cost and causing the break-even gate
to reject most history-collapse opportunities (283/391 = 72% of measured
ok rows came back history_reason='not_profitable').
Resize one render.test.ts fixture (6060 → 4848 chars) so the 'low cpt
unlocks a previously-rejected slab' assertion still exercises the
intended before/after edge under the new default.
Earlier commit a0f282f "corrected" Opus 4.7 rates downward to $2.50
input / $12.50 output based on a misread of the pricing source. Per
docs.claude.com/en/docs/about-claude/pricing (current), Opus 4.7 is:
Base input: $5 / MTok (same as Opus 4.5, 4.6)
Output: $25 / MTok
5m cache write: $6.25 / MTok
Cache hits/reads: $0.50 / MTok
Same dollar pricing as Opus 4.5 / 4.6 — what changed in 4.7 is the
tokenizer, not the per-token rate. The README now flags this so future
contributors don't assume hardcoded chars-per-token / image-token
constants tuned on earlier models are still safe on 4.7; only
count_tokens against the target model is honest.
Touches:
- src/dashboard.ts: ASSUMED_INPUT_USD_PER_MTOK back to 5.0, with a
long comment block citing the source and warning about tokenizer
differences. Updated card subtext on the headline accordingly.
- README.md: pricing table back to $5/$25/$6.25/$0.50, with a
callout that the tokenizer changed in 4.7.
When the cacheable-prefix count_tokens probe failed but the full-body
probe succeeded, the dashboard fell through to `baselineCacheableTokens
?? 0`. That silently treated the unproxied counterfactual as
cold-input-only — charging full $2.50/Mtok input rate on tokens that
demonstrably would have been cache-discounted at 10× cheaper. Net effect:
fabricated "$ saved" headline on every partial-probe row.
The fix:
1. New `TransformInfo.baselineProbeStatus: 'ok' | 'partial' | 'failed'`,
set by the proxy after awaiting both probes. 'ok' means either both
probes resolved OR the original body had no cache_control markers
(cacheable=0 is then exact, not estimated). 'partial' means the
full-body probe resolved but the cacheable-prefix probe didn't.
2. `tracker.ts` persists it as `baseline_probe_status` on the JSONL.
3. `dashboard.ts` gates its savings math on `status === 'ok'` (with a
back-compat path for legacy rows that pre-date the field). Partial /
failed rows now contribute to the request count but not to
baselineInputWeighted / actualInputWeighted / outputWeighted.
ocproxy's host-side dashboard already records baseline_probe_status in
its SQLite schema and is being updated in parallel to honor the same
gating semantic. See ocproxy commit fixing dashboard/src/lib/snapshot.ts
for the matching change.
The previous value was Opus 4.5/4.6 pricing. Opus 4.7 input is $2.50/Mtok,
output $12.50/Mtok. The dollar-saved headline was overstated by 2×.
OUTPUT_TOKEN_RATE (5×) and cache multipliers (1.25×/0.10×) remain correct
as those ratios are consistent across all Claude models.
Adds isCompressionProfitableAmortized() and a new
historyAmortizationHorizon option (default 1 = current per-turn
behaviour). When the host opts in to horizon ≥ 2, the history-collapse
gate evaluates expected lifetime cost over N future turns instead of
the cold per-turn comparison:
I × (CC + CR×(N-1)) < T × CR × N
where CC = 1.25 (cache_create), CR = 0.10 (cache_read)
At N=5 this accepts collapses with I/T < 0.30; at N=10 with I/T < 0.47.
The worst-case-for-image / best-case-for-text framing is on purpose —
we only collapse when the math wins under pessimistic warm-cache
assumptions, so a single-turn session can never pay a tax.
This is the 'try-then-decide-with-bounded-horizon' design from the
README's new 'unsolved part: multi-turn amortization' section. Picked
over JIT-style session state (rejected: introduces durable state),
always-collapse (rejected: dishonest per-request), and cache-bust-driven
(rejected: signal arrives one turn late).
- New option threaded through DEFAULTS, TransformOptions, PixelpipeOptions
- Both history call sites (early-exit helper + main success path) use the
amortized gate when horizon > 1, fall back to the cold gate otherwise
- 4 new unit tests covering fallback, accept-on-long-history, reject-on-
tiny-text, and the subset-relationship between cold and amortized gates
- 298 tests pass
Pixelpipe's static-slab gate returns early when the system slab is below
minCompressChars or fails the slab break-even check. Both exits skipped
the always-on Variant C history-image collapse, so production traffic
with a tiny system prompt but a huge messages[] array (e.g. Codex with
no project instructions / CLAUDE.md / skills) never got any compression
even though the dynamic side was massive.
Trace on a real-shaped request (system=29 chars, 56 message turns,
~275k outgoing text chars) before this change:
applied=false, reason=below_min_chars, imageCount=0, collapsedTurns=undefined
After:
applied=true, reason=applied, imageCount=20, collapsedTurns=56,
collapsedChars=256832, outgoingTextChars=19782
Implementation: extracted runHistoryCollapseAndFinalize() helper that
runs collapseHistory + countOutgoingTextChars + JSON.stringify. The two
early-exit paths now call the helper; if history collapses they flip
info.compressed = true (library.ts then returns reason='applied'),
otherwise they return the original body unchanged (preserving the
existing below_min_chars / not_profitable diagnostics for the slab).
Main success path unchanged — it already ran history collapse after
slab compression. No new flags, default-on for Opus 4.7 per host
gating. 294 tests pass.
Adds the data collection half of the adaptive chars-per-token plan
without touching the gate. Every isCompressionProfitable call site
now credits its chars to a named bucket on TransformInfo, and those
counts flow through toTrackEvent into the persisted JSONL.
transform.ts:
- New BucketName union: 'static_slab' | 'reminder' |
'tool_result_{structured,log,prose}' | 'history'
- bumpBucket() helper lazily allocates info.bucketChars
- toolResultBucket() routes by classifyContent shape so the post-
compaction text we credit matches the shape the gate evaluated
- Bump sites: static slab (combinedRaw), reminder (reminderRaw),
tool_result single-string (innerRaw), tool_result multi-block
per TextBlock (innerTextRaw), history (collapsedChars)
- Surfaces historyTextChars separately so history's no-collapse
rejections still contribute a denominator
tracker.ts:
- TrackEvent gains bucket_chars?: Partial<Record<BucketName, number>>
and history_text_chars?: number
- toTrackEvent copies them when populated, drops empty bucket maps
so happy-path events stay small
tests/tracker.test.ts:
- Asserts snake_case field names and nested shape land correctly
- Asserts absent buckets and pass-through events stay clean
No gate logic changed. CHARS_PER_TOKEN / SLAB_CHARS_PER_TOKEN /
HISTORY_CHARS_PER_TOKEN still drive isCompressionProfitable. Phase 2
will fit per-bucket slopes from the captured events and wire a
bucket-aware override.
docs/ADAPTIVE_CPT_PLAN.md: full plan + empirical basis (slab-only
regression on 87 events recovered cpt≈1.50 vs the baked 2.5
constant — 40% on non-slab compressions ship today).
Verified: tsc --noEmit clean; 290/290 tests pass (+2 new);
build emits dist/node.js.
The SSE/JSON scanner from #22 was wiring up measurement on every event
but the dashboard wasn't showing the numbers. Now the token-equivalent
card's "show calculation" panel exposes the raw counts the scanner
produced (text_delta + thinking_delta + tool_use chars + redacted block
count) alongside events_with_measurement so the operator can weigh how
load-bearing the gap really is.
No new headline card — the gap belongs inside the existing total-bill
panel because it's a diagnostic on the total-bill number, not a new
metric the user has to learn. When events_with_measurement << requests
(scanner fell back on a lot of events) the operator can see it; when it
tracks requests the comparison is real.
Refs #22.
Old labels were cryptic:
- 'total bill (input + output×5) · input-only: 71.1%' → wrong formula
- the m_pct_math block showed saved/baseline×100 but the card
actually displays saved/(baseline+output×5)×100 — math didn't
match the displayed number
New labels:
- m_pct card sub-line: '÷ (input + 5×output) — output billed at
5× input rate · input-only: XX%'
- m_tokeq card sub-line: 'input + 5×output — output billed at
5× input rate'
- m_tokeq tick sub-line: 'baseline N · input + 5×output'
m_pct_math now matches the displayed number:
- shows share_of_bill = saved / (baseline_input + output × 5) × 100
- breaks out saved, baseline_input, output × 5, baseline_total,
share_of_bill, and input-only % as a labeled sub-line
- adds a 'why include output' note explaining the weekly-meter
framing the README already documents
Pure label/formula-block edit. No math or wiring changes. 288 tests
pass, tsc --noEmit clean.
Adds a third tee branch in createProxy that walks the response body and
counts Unicode code units across text_delta / thinking_delta /
input_json_delta / tool_use blocks, plus a redacted_thinking block count.
Same shape on streaming (SSE) and non-stream (content[]) responses.
Numbers are independent of Anthropic's usage.output_tokens - they give a
real ruler against the redacted_thinking-inflated bill that surfaced in
the May-2026 weekly-meter audit (#22).
Wiring:
OutputMeasurement type in proxy.ts (textChars, thinkingChars,
toolUseChars, redactedBlockCount)
teeForUsage now returns measurementPromise alongside usagePromise +
errorBodyPromise; Promise.all in handle() awaits all three
ProxyEvent gains optional measurement: OutputMeasurement
TrackEvent gains text_chars_measured / thinking_chars_measured /
tool_use_chars_measured / redacted_block_count_measured (optional,
non-breaking JSONL shape)
toTrackEvent() copies them through
Tests (6 new in proxy-usage.test.ts):
text_delta count across multi-event SSE
message_delta output_tokens overrides message_start placeholder
thinking_delta + redacted_thinking block count
tool_use input_json_delta reassembly
non-stream content[] walks same shape
5xx no-body path leaves measurement undefined
288 tests pass, tsc --noEmit clean, dist/node.js builds.
Dashboard surfacing lands in a follow-up so the wire-up stays reviewable.
Refs #22
The fifth headline card surfaces the absolute number Anthropic's weekly
meter actually sees: input + output×5 in input-token-equivalents. Pairs
with the baseline counterfactual in the sub-line so a heavy-output day
shows as a real spike instead of hiding behind a flat 73% gauge.
Grid CSS bumps to 5 columns with a middle breakpoint at 1200px so the
cards don't crush at typical widths. No new computation — both fields
already lived in the /proxy-stats payload from #20.
The old single-rate weight (cr>0 ? 0.10 : cc>0 ? 1.25 : 1.0) collapsed
the entire cacheable prefix to one class. On warm turns the proxied path
bills cc * 1.25 for the new user-typed tail AND cr * 0.10 for the rest
of the prefix - but the formula attributed 100% to cr * 0.10, making the
unproxied path look ~12.5x cheaper than reality and producing negative
'saved' on every warm row.
The corrected per-event break-even (no compression, identical content):
cold start (cr=0, cc>0): ccU = cacheable, crU = 0
warm turn (cc>0, cr>0): ccU = min(cc, cacheable), crU = cacheable - ccU
no caching (cc=0, cr=0): ccU = 0, crU = 0
cache-read (cr>0, cc=0): ccU = 0, crU = cacheable
baseline_eff = ccU*1.25 + crU*0.10 + coldTail*1.0
Centralized as computeBaselineInputEff() and computeActualInputEff() in
src/core/baseline.ts, used by the live dashboard (dashboard.ts), the
JSONL replay path (dashboard.ts), and per-session rollup (sessions.ts).
Verified on the May-2026 regression (7 events, mixed cc/cr, not negative):
pre-fix sum = -9,786 'saved' tokens, post-fix sum = +19,452 tokens,
matching the per-event compression delta (~+2,780/event) exactly.
Aggregate impact on the 914-event corpus: 'saved %' moves from 73.1% to
73.2% - the headline barely shifts because warm turns are a minority,
but per-event dashboard rows no longer show false negatives.
282 tests pass, tsc --noEmit clean.
Anthropic's /v1/messages usage block returns more fields than we were
capturing. The flat 4-field view (input_tokens, output_tokens,
cache_creation_input_tokens, cache_read_input_tokens) historically dropped:
- cache_creation.ephemeral_5m_input_tokens (1.25x rate price tier)
- cache_creation.ephemeral_1h_input_tokens (2x rate price tier)
- server_tool_use.web_search_requests (billed per-request, not per-token)
Extends:
- Usage interface (types.ts): nested cache_creation + server_tool_use
- TrackEvent (tracker.ts): cache_create_5m_tokens, cache_create_1h_tokens,
web_search_requests as optional fields
- toTrackEvent copy logic: passes through when defined, omits when absent
(a real 0 stays 0 - the dashboard needs to tell "missing" from "zero")
Also adds .gitignore entries for draumode MCP scratch files
(*.draumode.ts, *.excalidraw) so exploratory diagrams don't leak into
the repo.
Pure data collection. No behavior change. Sets up the dashboard fixes
(tasks #20 and #21) where the data will actually get rendered.
282 tests pass, typecheck clean, build clean.
Fragility #3 from end-of-session audit. The previous gate tests pinned
isCompressionProfitable(slab, 100, 2, 2.5) === true against 'A'.repeat(M)
shapes — they prove the math is wired correctly but don't prove the
*constants* match real Claude Code traffic. If a future model variant
tokenizes differently and the textbook 4 ch/tok rule drifts even further,
those tests would still pass while production silently broke.
Anonymized 6 production shapes from events.jsonl (2026-05-19..2026-05-20):
- PRODUCTION_SLAB_161K (multi-col, 52 chars/row, 11 ings/141 lines/2 cols)
- PRODUCTION_SLAB_135K_DENSE (neuline-heavy, 19 chars/row, 24 ings)
- PRODUCTION_SLAB_169K_HEAVY (very dense, 16 chars/row — actually rejected)
- BELOW_MIN_CHARS_TINY (142 chars, pre-filter)
- BELOW_MIN_CHARS_BORDERLINE (1123 chars, pre-filter)
- HISTORY_COLLAPSED_LONG_SESSION (collapsed path captured)
Each fixture stores its real captured decision plus the shape parameters
(origChars, numCols, approxCharsPerRow). synthesizeText() pads 'A'.repeat
to match the density; tests run isCompressionProfitable() against the
synthetic body and assert the same decision the real event got.
\#\# Notable: 135K_DENSE divergence
The real 135K neuline-heavy event was ACCEPTED (compressed=true) but the
synthetic 'A'.repeat(19) reconstruction REJECTS even at cpt=2.5. Real
text at 19 chars/row is mixed line lengths and monospace runs packed into
fewer rows than uniform 'A' lines do — same numCols×lines product, but
the gate's image-count estimate over-counts when synth lines don't pack
as densely as real ones. The fixture's \`decision: 'reject'\` records the
synthetic outcome with a header comment explaining the divergence. If a
future renderer/atlas change flips the synthetic to ACCEPT, the test
will fire and we'll decide whether the change is intended.
\#\# Export changes
SLAB_CHARS_PER_TOKEN and HISTORY_CHARS_PER_TOKEN exported from transform.ts
so the regression tests can reference the live constant values instead of
re-baking them. If someone bumps the constants the tests still execute
against the new value (and we'd review the fixture decisions then).
\#\# Tests
8 files / 280 tests pass, typecheck clean, build clean. 7 new it() blocks
cover the 5 captured shapes + the synthetic divergence note.
The override-gate previously used `o.charsPerToken !== CHARS_PER_TOKEN`
to detect "host pinning". After CHARS_PER_TOKEN=4 was replaced with
slab/history-specific 2.5 constants, a host that deliberately passes the
literal `4` (the same as DEFAULTS) was silently swapped to 2.5 - the
gate couldn't distinguish "host passed nothing" from "host passed 4".
Fix: check the raw opts (\`opts.charsPerToken !== undefined\`) instead
of comparing the merged value against a sentinel. Host-supplied `4` now
flows through as `4`, even though it matches the default.
Tests pin both directions:
- slab path (render.test.ts): explicit cpt=4 keeps the 161k slab
REJECTED at cpt=4 (would be ACCEPTED at the built-in 2.5)
- history path (history.test.ts): explicit cpt=4 keeps a 14-turn closed
conversation REJECTED at cpt=4 (would collapse at the built-in 2.5),
with companion cpt=2.5 test proving the fixture actually straddles
275 tests pass, typecheck clean, build clean.
Production telemetry (N=10 events with history_reason="not_profitable",
2026-05-18..05-20) shows body-level chars-per-token on rejected history
events: median 1.09, max 1.10. The gate at cpt=4 was estimating text as
3.7× cheaper than reality on every history-collapse decision in the
sampled traffic — same shape of bug as the slab path fixed in c36ec3f.
Why this matters more than the slab fix:
• The slab is cached (cache_create×1.25 once, cache_read×0.10 after).
Slab compression saves bytes mostly on cold misses + 2k tok/warm hit.
• History content is NOT cached. Once turns leave the cache breakpoint
budget (4 max per request), every subsequent request pays 100% for
them. A long agentic session re-bills the same closed-prefix turns
on every turn — exactly the workload Variant C was designed to
compress, and exactly the path the gate was silently rejecting.
Apply same fix as c36ec3f: HISTORY_CHARS_PER_TOKEN=2.5 (upper bound of
empirical data, identical safety property to SLAB_CHARS_PER_TOKEN since
history content shape ⊆ slab content shape in tool-heavy sessions).
Host can still override via TransformOptions.charsPerToken. The
override-gate `o.charsPerToken !== CHARS_PER_TOKEN` means unspecified
requests (DEFAULTS.charsPerToken=4) get the empirical cpt while explicit
non-default host values win — same intentional pattern as the slab path.
Test: regression fixture pinning a 14-turn × 1500 chars/turn conversation
where:
• cpt=4 path → 'not_profitable' rejection (image cost 5k > text 4k)
• cpt=2.5 path → 'collapsed' with 10 turns merged (text 6.4k > image 5k)
Counterfactual half uses cpt=4.5 to bypass the override-gate (passing
exactly 4 collides with the DEFAULTS sentinel).
273 tests pass, typecheck clean, build clean.
Production data (N=354 cold-miss count_tokens probes, 2026-05-18..05-20):
real body-level chars-per-token is 1.17 median, 2.62 max — never near
the English-prose 4 baked into CHARS_PER_TOKEN. The gate has been
estimating text-token cost at 3.4× cheaper than reality and silently
rejecting every realistic Claude Code system slab.
Concrete production case (orig_chars=161,101, multi-col=2):
• OLD gate (cpt=4): text = 161101/4 = 40,275 tok
image = 8 × 5500 = 44,000 tok
→ reject (margin: -3,726)
• NEW gate (cpt=2.5): text = 161101/2.5 = 64,440 tok
image = 8 × 5500 = 44,000 tok
→ ACCEPT (margin: +20,440)
• Reality (cpt=1.17): text = 137,694 tok, savings = 93,694 tok/request
SLAB_CHARS_PER_TOKEN=2.5 is the upper bound of observed cpt across the
sample — picking the upper bound keeps the prime-directive guarantee
intact: the text-token estimate is a LOWER bound on real text cost, so
any `imageCost < textTokens` decision is also `imageCost < realTextCost`
for any future workload with cpt ≤ 2.5.
Scope: this is slab-specific. Reminders and tool_result content have
unknown shape (could be raw English prose with cpt~4), so those gate
call sites still use the conservative CHARS_PER_TOKEN=4. Host can
override per-request via TransformOptions.charsPerToken (e.g., to plug
a live empirical fit in front of the static default).
Tests: 2 new regression tests pinning (a) the production-shape 161k slab
compresses end-to-end without an explicit cpt override, and (b) the gate
math at cpt=2.5 vs cpt=4 produces accept-vs-reject on the same input.
272 tests pass, typecheck clean, build clean.
Production data (2026-05-20): real 161,101-char system slab being rejected
by the row-aware profitability gate as `not_profitable`. multiCol=2 packs
282 rows/image, but 2000+ newline-bounded lines need ~10 images, and at
2500 tok × 10 × 1.10 = 27,500 tok the gate (correctly) refuses since the
text baseline is cheaper.
Fix: strip per-line trailing whitespace and collapse 3+ consecutive
newlines to exactly 2 *before* feeding the renderer. Both transforms are
lossless for the model (whitespace at line-end carries no signal; blank-
line runs longer than 2 are visual noise). This drops visual-row count
without changing meaning, flipping borderline-rejected slabs profitable.
Accounting uses raw (pre-compaction) length for `origChars` and
`compressedChars`, so the savings credit reflects what Anthropic would
have billed — not the post-compaction figure.
Applied at four sites:
- combined static slab (system + tool docs)
- system-reminder block
- tool_result string content
- tool_result array-of-blocks (joined text)
8 new unit tests for `compactSlabWhitespace`: empty input, trailing
whitespace stripping, 3+ newline collapse, paragraph-break preservation,
indent preservation, idempotency, realistic markdown shrink.
270 tests pass, typecheck clean, build clean.
Rip the compressHistory opt-in and run Variant C history-image collapse on
every request that has a closed-prefix run. There is now ONE codepath: no
toggle, no env switch, no per-request opt-in. The reason the option was
defaulted-off was a -250% live measurement on 2026-05-19 (commit c3506e3 era);
that wasn't a "marginal upside" problem, it was a math bug. This commit
fixes the bug and turns the feature on.
Root cause of the -250% loss
----------------------------
`isCompressionProfitable()` has two paths: a row-aware path (string input,
matches `renderTextToPngs` image budgeting exactly) and a looser chars-only
path (number input, assumes dense lines). History.ts was calling it with
`text.length` (number) → loose chars-only estimate. History text is
newline-heavy (`--- role ---` headers, JSON arg blocks, [tool_use] /
[tool_result] labels), so the chars-only estimate under-counted images by
5-10× and let net-losers through the gate.
Secondary contributor: tool_use args were serialised with
`JSON.stringify(input, null, 2)` — pretty-printed, one short row per JSON
field. At cols=100 each field claimed a full row instead of wrapping at
~100 chars, inflating image count for tool-heavy histories.
Changes
-------
- src/core/history.ts
- `ProfitableFn` signature: `(text: string, cols: number)` (was `textLen: number`).
Documented the asymmetry so callers can't regress this.
- `blocksToText` tool_use: compact `JSON.stringify(input)`, no indent.
- Pass `text` (not `text.length`) into the profitability check.
- src/core/transform.ts
- Delete `compressHistory`, `historyKeepTail`, `historyMinPrefix` from
TransformOptions and DEFAULTS.
- History collapse is now unconditional when `messages[]` is non-empty.
- Wrap the gate in a closure that pins `numCols=1` + per-request `cpt`,
so the gate's row-aware decision is byte-identical to the single-col
renderer's actual image count.
- src/worker.ts: drop `COMPRESS_HISTORY`, `HISTORY_KEEP_TAIL`,
`HISTORY_MIN_PREFIX` env vars from `Env` + transform options.
- src/node.ts: stale comment refresh — no toggles, period.
- tests/history.test.ts:
- Rename describe "transformRequest + compressHistory" →
"transformRequest history compression (always-on)".
- Remove every `compressHistory: true, historyKeepTail: X, historyMinPrefix: Y`
from transformRequest calls — options no longer exist.
- Bump fixtures for default `keepTail=4` / `minCollapsePrefix=10`
(14-turn body, 3500 chars/turn) so the now-honest row-aware gate
accepts. Renamed "8-closed + 2-live" → "10-closed + 4-live" to match
the new defaults.
- Bump straddle-test per-turn body 2500→3500 chars: tool block labels
add ~65 chars of header overhead that pushed the tighter fixture
under the row-aware boundary.
- Rename "JSON-pretty args" test → "COMPACT JSON args"; assert no
pretty indentation.
Verification
------------
- npm test → 263/263 green
- npm run typecheck → clean
- npm run build → dist/node.js builds clean
Production runs multiCol=2 by default. The pre-flight gate
`isCompressionProfitable()` was using a flat TOKENS_PER_IMAGE=2500
constant at every numCols, so it believed multi-col images were ~2x
cheaper than they actually are and would let net-loss compressions
through on wider canvases.
Replace the flat constant with `effectiveTokensPerImage(numCols)`:
numCols=1: 2500 (empirical, N=33 cold-miss measurement)
numCols>1: 2500 * n * 1.10 (linear extrapolation + 10% margin)
The 10% margin absorbs extrapolation noise — we don't yet have
empirical cost measurements at numCols>=2. Per user constraint
"ok pre-flight gate is ok to be not correct as long as we don't
lose money," every uncertainty resolves in favor of pass-through.
At worst we miss opportunities; we never burn money on net-loss
images.
Test fixtures bumped to match the new break-even (~22k chars at
n=2). Two obsolete cpt-flip tests REWRITTEN to assert the safe
behavior: cpt override flips at numCols=1, but at numCols=2 the
safety margin holds and the slab stays rejected.
Old break-even at default cpt=4:
n=1: ~10k chars (1 image @ 2500 tok == 10k/4 = 2500 text tok)
n=2 [old/wrong]: ~10k chars (same flat cost)
n=2 [new/safe]: ~22k chars (1 image @ 5500 tok == 22k/4 text tok)
Verified: 263/263 tests pass, typecheck clean, build clean.
The baseline_tokens probe alone reports the COLD cost of the unproxied
body — but the user's request actually carries `cache_control` markers
that would have cached on the unproxied path too. Comparing cold-
baseline against warm-actual was an apples-to-oranges inflation of
saved%.
Fix: fire a SECOND count_tokens probe on the body TRUNCATED at the last
`cache_control` marker. Difference between the two is (cacheable_prefix,
cold_tail). Dashboard weights the prefix by the SAME cache class the
actual response landed in:
cacheable = baseline_cacheable_tokens or 0
cold_tail = baseline_tokens − cacheable
weight = cr>0 ? 0.10 (warm hit — both paths cache)
: cc>0 ? 1.25 (cold create — both paths build cache)
: 1.0 (no caching — marker rejected or absent)
baseline_input_eff = cacheable·weight + cold_tail·1.0
actual_input_eff = input + cache_create·1.25 + cache_read·0.10
saved = baseline_input_eff − actual_input_eff
Now the headline number is the exact unproxied counterfactual, not a
cold-every-time approximation. No estimation, no α, no assumptions —
every input is a measurement.
Wiring:
- src/core/proxy.ts: buildCountTokensCacheablePrefix() walks messages
backward → system backward → tools backward (reverse cache order) for
the last cache_control marker, truncates, attaches a sentinel user
message when needed. Second probe fires in parallel; null/missing
leaves the field absent.
- src/core/transform.ts + tracker.ts: new baselineCacheableTokens /
baseline_cacheable_tokens fields, emitted only when > 0.
- src/dashboard.ts: per-event cache-aware math in update() and
replay(); Totals renamed to baselineInputWeighted; /proxy-stats
payload field is `baseline_input_weighted`.
- src/sessions.ts: per-session tokensSavedEst uses the same formula so
session and global rollups agree.
Tests: 2 new in proxy-usage covering (a) body with cache_control →
two probes fire with different bodies and both numbers land on info,
(b) cacheable-prefix probe failure doesn't poison the full probe.
260→262 tests, all green; typecheck and build clean.
The headline saved%/saved$ numbers are now MEASURED, not estimated:
baseline = sum of /v1/messages/count_tokens(originalBody).input_tokens
actual = sum of (input + cache_create×1.25 + cache_read×0.10)
saved = baseline − actual
Output is excluded from both sides — it's identical with or without
compression (same prompt → same response), so it cancels in the
numerator and would only drag the percentage toward zero. The proxy
now makes one parallel side call to /v1/messages/count_tokens (free,
no billing) before forwarding /v1/messages, and the result lands on
TransformInfo.baselineTokens → TrackEvent.baseline_tokens → the
DashboardState rollup.
Removed (~280 lines of dead code):
- FALLBACK_ALPHA, FALLBACK_ALPHA_LOW, FALLBACK_ALPHA_HIGH constants
- estImageTokens(), baselineCost(), effectiveCost() helpers
- FitSample interface + FIT_SAMPLE_CAP + fitSamples ring buffer
- DashboardState.fitCosts() (~218 lines of OLS regression)
- Three-mode ladder (joint / constrained / stale) and all UI for it
- saved_pct_low/high, saved_effective_tokens_low/high range fields
- renderSavingsMath() range/regime branches
- m_pct_regime DOM node + its color/text wiring
- cost_fit payload field + all frontend consumers
/proxy-stats payload is now: requests, compressed_requests,
baseline_input_tokens, actual_input_weighted, saved_input_tokens,
saved_pct, saved_usd, pricing_assumptions, uptime_sec,
compression_enabled. Nothing else.
The recent-requests table now shows baseline / actual / saved per
row (input-side, matching the cards) instead of the old img-tok
estimate + effective_actual (which mixed output × 5 in inconsistently).
Sessions panel: per-session tokensSavedEst now uses the same real
baseline−actual formula. Events without both a count_tokens probe
AND an upstream usage block don't contribute to the rollup — no
synthetic 4-char/tok fallback.
Tests:
- dashboard-fit.test.ts: deleted (fitCosts is gone)
- proxy-usage.test.ts: inverted the privacy regression — count_tokens
is now explicitly expected as the one whitelisted side path; added
baselineTokens-attachment and count_tokens-failure-survives tests
- sessions.test.ts: rewrote the two savings tests against the new
baseline_tokens + usage formula
Sending the pre-compression body to /v1/messages/count_tokens means
Anthropic sees the original text, which is exactly what pixelpipe is
built to prevent. Probe results were 'free' in dollars but not in
privacy; the whole point of imaging the slab is that Anthropic does
NOT receive the slab text.
Removed:
- countTokensUpstream + buildCountTokensBody in src/core/proxy.ts
- measurePromise wiring + finalize await
- baselineTokensMeasured / actualTokensMeasured on TransformInfo
- baseline_tokens_measured / actual_tokens_measured on TrackEvent
- measuredEvents / effectiveCostActualMeasured / *Baseline*
fields on DashboardState
- measured_events / saved_pct_measured / effective_cost_*_measured
/ saved_effective_tokens_measured on /proxy-stats payload
- HTML headline branch + 'show calculation' rows for measured path
- 2 dashboard-fit tests asserting the measurement-path behavior
Added regression test: proxy must NOT hit any upstream path other than
/v1/messages.
Real-savings path now: self-derive α from billed usage + own char
walker. Coming next.
count_tokens accepts only model/messages/system/tools/tool_choice/thinking/
mcp_servers. Forwarding the verbatim /v1/messages body included stream,
max_tokens, temperature, metadata etc, which the endpoint rejected with
400. Probe returned null silently → measured_events stuck at 0.
Strip to the accepted fields before posting. Now both probes succeed,
the dashboard headline can use saved_pct_measured = (Σ baseline −
Σ actual) / Σ baseline — exact, no regression, no priors.
Diagnostic logging kept on the probe path: failures surface in stderr
instead of disappearing.
teamchong