Router was calling dashboard.servePng() with no argument, so every per-id
thumbnail URL fell back to 'latest image' and all N thumbs on a row
rendered the same PNG. Forward the parsed id from the query string.
Pixelpipe now always uses the full 1568px canvas (cols=313) and packs up
to 50,000 chars per image. The multi-column code path and the
shrink-to-content path are no longer used by transform.ts - both were
sacrificing token savings on dense content to make sparse content marginally
more readable, and the readability gain wasn't real (the renderer's
output was unreadable in either layout once the content was actually dense).
Key changes:
- READABLE_CHARS_PER_IMAGE: 6_000 -> 50_000
- DEFAULT_COLS: 100 -> 313 (full 1568px / 5px cell)
- shrinkColsToContent() is now a no-op (returns cols unchanged)
- MinToolResultChars / MinReminderChars default to 50k (was 6k)
- transform.ts always renders single-column at full canvas
Practical impact: a 6 KB tool_result that previously rendered as 4 narrow
508x488 pages (1324 image tokens, 86% of text cost) now renders as 1
full-canvas 1568x488 page (~331 image tokens, 22% of text cost) - 4x
more savings on the same content with no quality change.
Rebuilt dist/. Tests: 315/315 pass.
SessionSummary: divide saved_$ by baseline_input_$ (was actual_input + output,
which produced 719% on cheap sessions). Math now matches the user-stated
analogy: baseline=10, actual=5 -> saved=5, 50%.
RecentRequests: stamp full imagePngs[] in TransformInfo so each rendered PNG
gets a ring-buffer id. Row shows a strip of 28px thumbs (one per image)
instead of a single 'view' button, so multi-image requests are visible at
a glance. Thumbs are <button>-wrapped for keyboard accessibility.
After the new atlas geometry (CELL_W=5, CELL_H=8, LINES_PER_IMAGE=195,
MaxCharsPerImage(100)=19500) and the WIP shrinkWidth=true default in
transformRequest, a single 100-col image now holds ~19.5k chars of dense
ASCII at cpt=4 (vs ~15.6k previously). Combined with the gate hoisting
the width-shrink before prediction, the existing test fixtures (which
mostly used 'a'.repeat(N) with no newlines) now compress profitably at
every length above MIN_COMPRESS_CHARS=2000.
Per user direction ("use the actual gate function to compute new
expected values", "prefer rewrite over delete unless premise is
genuinely gone"):
- render.test.ts: 19 isCompressionProfitable() expectations rewritten.
Numeric-literal callers (which used a removed back-compat overload)
converted to 'a'.repeat(N) strings. Per-test comments updated to
describe the new break-even framing under MaxCPI(100)=19500. Real-shape
regression fixtures (PRODUCTION_SLAB_135H_DENSE, 169H_HEAVY) now
ACCEPT under the larger atlas; comments note the conservative-side
shift no longer rejects these dense shapes.
- history.test.ts: 5 expectations rewritten. The amortized-horizon and
prior-warm-tokens gates no longer reject at the historical fixture
sizes because image side cost has dropped well below text side cost
even at cpt=4.5. Reasons updated from 'not_profitable' to 'collapsed'
where the gate now collapses cleanly; amort() and cold() calls flipped
from .toBe(false) to .toBe(true) with rationale.
src/ untouched. Final count: 315 passed, 7 skipped, 0 failed (was 24 failing).
npx vitest run: 100% green.
The system slab's break-even gate was estimating image cost at the full
configured cols width while the renderer was shrinking the canvas to the
longest wrapped line. Gate and renderer now agree: shrink once up front,
pass the shrunk cols to both evalCompressionProfitability and
renderTextToPngsMulticol.
Two fixes to the image-cost prediction that gates compression:
1. **Content-aware image cost.** The old gate used a fixed
`TOKENS_PER_IMAGE_SINGLE_COL = 2500` constant per image, modelling
every tool_result block as a full 1568px canvas. The renderer
actually produces images sized to the wrapped line count, so a
5,000-char tool_result becomes a ~80px-tall image costing ~54 tokens,
not 2,500. Result: the break-even for small blocks was over-estimated
by ~50x and the gate rejected anything below ~10k chars even though
image-form was cheaper.
Replace `estImages × effectiveTokensPerImage(numCols)` with a single
content-aware `imageTokensCost(text, cols, numCols, imageCountCap)`
that mirrors the renderer's height math:
`tokens = (rows + 1) × overhead + fullImageRows × perRow`, derived
from the cold-miss anchor and applied row-proportionally for partial
final images. Calibrated against N=33 production cold-miss
measurements (median multiplier 1.05, mean 1.32 — high-image case is
~1.05x of the documented `width × height / 750` rate).
2. **Width-shrinking on per-block paths.** For per-block call sites
(tool_result, reminder) the renderer now narrows the canvas width to
the longest wrapped line via `shrinkColsToContent`. Multi-col
layouts already pack tightly; this captures the single-col savings
that were previously paying for whitespace. The gate threads
`shrinkWidth` through to `evalCompressionProfitability` /
`isCompressionProfitable` / `isCompressionProfitableAmortized` so
the prediction matches what the renderer will actually produce.
System-slab and history-collapse paths pass `shrinkWidth=false` —
they fill the configured canvas deliberately.
Knobs removed: `MinToolResultChars` and `MinReminderChars` now
default to 0 (let the math decide). The TransformOptions fields are kept
for env-override observability but no longer set a floor in production.
`isCompressionProfitable` / `evalCompressionProfitability` /
`isCompressionProfitableAmortized` are now string-only at the public
API; the numeric-length fallback is gone (it under-counted images and
silently leaked net-losing compressions through).
Tests: gate-flap and history.test updated for the new physics, but 24
tests in render.test.ts still assert the OLD over-estimation as truth
(e.g. "5,000 chars is not profitable", "7000 chars stays as text").
They need to be deleted or have their expected verdicts inverted — left
for the next session to finish the rewrite cleanly. Typecheck passes.
Measured impact: the production tail's $1.43 flap is gone and per-block
tool_result/reminder content is now genuinely compressed when it would
save tokens, not just when it crosses the (wrong) 10k-char threshold.
Add per-path actual-billed cost counters to the dashboard alongside the
existing counterfactual baseline. The compressed-vs-passthrough split
sums what each code path actually cost on real traffic, partitioned by
`info.compressed`, with no probe gating or counterfactuals. Surfaces
sample counts in the UI so operators can judge sufficiency rather than
having the dashboard auto-claim significance.
Headline math also switched to share-of-all-paid-spend (counterfactual
denominator bounded at 100%) so the slice number never lies when
pixelpipe didn't run on every request. Regenerated dashboard-bundle.
The previous patch divided savings by all-rows ACTUAL spend, which is
not bounded — pixelpipe shrinks the denominator, so a single big
cold-miss compressed request showed '280% saved'. That's nonsense:
you can't save more than you would have paid.
Correct denominator is what the user WOULD have paid with no pixelpipe:
measured rows contribute their cache-aware baseline, unmeasured/probe-
failed/passthrough rows contribute their actual (pixelpipe either
didn't run there, or we can't measure the counterfactual, so actual ≈
baseline). Bounded at 100% by construction.
- Totals: rename allActualInputWeighted role to diagnostic only, add
allBaselineEquivalentWeighted as the headline denominator input.
- update(): when haveBaseline, add baselineInputEff to the counterfactual
pool; otherwise add actualInputEff. Output × 5 unchanged on both sides.
- serveStats(): pctAllSpend = saved / (allBaselineEquiv + allOutput).
- types.ts + StatsHeader.svelte: surface all_baseline_equivalent_weighted,
rewrite the explainer math.
Single-request smoke (the case that surfaced the bug): saved=397k
input tokens, baseline_equivalent≈400k, output_equiv tiny → ratio ≈
99% saved, which is honest (one cold-miss compressed turn on an
otherwise empty session).
The 'share of total bill saved' headline divided savings by
baseline_input_weighted, which only accumulated on rows where the
count_tokens probe succeeded AND the request had upstream usage. Every
passthrough turn, every probe-failed turn, and every flap-polluted
untransformed turn was excluded from the denominator. That's a
cherry-pick: it answers 'did pixelpipe help on the rows where it ran'
instead of 'did pixelpipe move my real bill'.
Track allActualInputWeighted / allOutputWeighted ungated on probe
success (any request with a usage block contributes), expose
saved_pct_of_all_spend on /proxy-stats, and rewrite the headline card
to use it. Numerator is still measured-rows-only because that's the
only place we have a counterfactual baseline; you can't fabricate
savings for rows where the probe failed. But the denominator now
matches the user's actual bill.
The legacy saved_pct_of_total_bill stays on the wire for back-compat
(marked deprecated in types.ts). The headline card no longer reads it.
Card flips color: positive % stays green, negative % goes red, because
flap-driven net losses are real and should be visible.
Production data (ocproxy claude-opus-4-7, 2026-05-23 09:19–09:21) showed
three-turn sessions still flapping AND paying cache_create on every
turn despite the symmetric burn shipped in 0cfe08a:
09:19 APPLIED cc=156k cr=6.7k $0.99 cold start
09:20 declined cc=181k cr=0 $1.23 FLIPPED to text
09:21 APPLIED cc=156k cr=6.7k $1.01 FLIPPED back to image
The slab gate's gateEval said image was profitable by 154k margin at
the flip turn, but applied=0 — the slab gate had decided to stay in
image mode, then the history-collapse gate (which got hardcoded 0 for
priorWarmTokens AND priorWarmImageTokens) flipped the verdict.
Wire o.priorWarmTokens + o.priorWarmImageTokens through to both
history-collapse call sites so the symmetric anti-flapping burn
applies to every gate that can change the session's bill shape, not
just the slab. Foundational follow-through on the same fix.
Production data (ocproxy Opus 4.7, 2026-05-22 23:15–23:44) showed the
slab break-even gate ping-ponging between text and image modes within a
single session, paying cache_create on whichever side it just flipped
away from. Two flips in 30 minutes cost ~$0.71 + ~$0.40 of avoidable
cache_create on a single conversation.
Root cause: priorWarmTokens was asymmetric. It penalised compressing
when the un-rewritten text path was warm, but did not penalise
decompressing when the rewritten image path was warm. Per-turn cost
favored flipping; the flip forced cache_create on the new side; next
turn flipped back.
This change adds the symmetric counterpart:
text→image flip: warm text invalidated → burn applied to IMAGE side
(existing behavior, priorWarmTokens)
image→text flip: warm image invalidated → burn applied to TEXT side
(NEW, priorWarmImageTokens)
Once a session commits to a mode, staying in that mode is cheaper than
flipping by the burn cost. Mode changes only happen when the per-turn
delta exceeds the burn, which is exactly when the flip is actually
worth the cache_create. Foundational fix — no thresholds.
Observability:
- New exported `evalCompressionProfitability()` decomposes the slab
gate's evaluation into imageTokens, textTokens, burnImageSide,
burnTextSide, and verdict. Same constants and helper as the gate
itself, no drift risk.
- New `TransformInfo.gateEval` field stamps that decomposition into
every applied or not_profitable event so hosts can persist the verdict
margin and prove flap-prevention efficacy from telemetry.
Back-compat:
- New params default to 0; single-arg callers behave identically.
- 315 existing tests unchanged; +7 new tests pin the new semantics
(back-compat, asymmetric ⇒ symmetric, both-sides-warm cancellation,
eval verdict matches gate verdict, burn term math).
- .npmrc: pin minimum-release-age=4320 (3 days) for supply-chain
defence; new versions can't be installed until they've been
on the registry long enough for typo-squat / compromise to be
caught and yanked.
- minimum-release-age-exclude=@cloudflare/* so first-party
packages don't block our own iteration.
- pnpm.overrides forces ws>=8.20.1 transitively via miniflare,
closing GHSA-58qx-3vcg-4xpx (moderate, uninitialized memory
disclosure in ws). Dev-only chain, no prod runtime impact, but
worth keeping clean for dependabot.
The reflow-inimage finding (commit bb9c231) made 5×8 viable, so
DEFAULT_CELL_W_BONUS / DEFAULT_CELL_H_BONUS reverted to 0 — but many
comments and the README still said 7×10 was production.
Touched:
- README.md — eval table, density bullet, sweep narrative,
cell-pitch section, measurement header,
Limitations section
- src/core/render.ts — RenderStyle docstring + cell-bonus docstrings
+ multicol intro comment
- src/core/transform.ts — TransformOptions.reflow docstring, DEFAULTS
comments (minReminderChars floor rationale,
maxImagesPerToolResult cap, reflow gate
rationale, anchor scaling formula, LINES_PER_IMAGE
worked example)
No code behavior changed — every constant still resolves through CELL_W /
CELL_H, and the eval harness still overrides the cell via cellWBonus /
cellHBonus when comparing variants. The 5×8 production path is what was
already shipped on 81550f6 (reflow + grayscale + L1/L2 harness) and
bb9c231 (reflow-inimage).
Co-render the OCR instruction into the same PNG as the content,
delimited by '===…===' bands, with the API system field dropped.
L1 OCR fidelity, Opus 4.7, 20 production blocks, 7×10 cell:
- baseline (text-only): 97.91% mean / 96.25% min
- reflow (separate system): 91.99% mean / 82.59% min (-5.93pp)
- reflow-inimage: 98.95% mean / 96.42% min (+1.04pp)
reflow-inimage wins on all 20/20 blocks vs reflow, and beats the
text-only baseline on 17/20 blocks. Three blocks hit 100%. The
-5.93pp reflow regression that the cell-pitch sweep partially
recovered disappears entirely when the instruction is co-rendered.
Mechanism: when system carries the instruction and the image
carries the content, the model does cross-modal binding to figure
out what the image is for. Co-rendering reduces it to a
single-modal task with an unambiguous parse rule.
Files:
- eval/eval-l1-ocr.mjs: add reflow-inimage variant + prompt
- README.md: new section before history compression
- eval/EXPERIMENT_LOG.md: attempt #2 writeup
- eval/results/{l1-report.md, l1-results.json}: regenerated
Pack text into a continuous sentinel-delimited stream (↵ = U+21B5) so
wrapLines fills every row to `cols` instead of leaving dead right-margin.
Adds reflow/dereflow, renderTextToPngsReflow{,MultiCol} variants, a full
test suite, and an A/B eval harness with L1 OCR + L2 session results.
Replace the hand-rolled dashboard with a Svelte component tree (App +
Sessions / StatsTable / LatestPng / RecentRequests / StatsHeader /
Cleanup / CompressionToggle / ToastTray), move the API layer into
lib/, and split state into stores/.
Add a dedicated dashboard build (`build:dashboard-ui` + a separate
tsconfig.dashboard.json) so dashboard sources typecheck independently
of the proxy core, which previously broke the root `tsc` build.
Snap the collapse cutoff to a `collapseChunk`-sized grid (default 50) so
the rendered history image stays byte-identical for a full window of
turns. That lets Anthropic's prompt cache `cache_read` the history
prefix (0.1x) instead of re-billing `cache_create` (1.25x) every turn.
The cutoff floors at `minCollapsePrefix` (clamped to the raw cutoff) so
short conversations still collapse without the boundary drifting.
Also log `history_image_sha8` per request -- a sha8 of the concatenated
history-image base64. An unchanged hash across consecutive collapsed
events is ground-truth proof the prompt cache is being hit; a hash that
moves every turn signals the cache-key drift regression is back.
Capture the practical conclusions from recent VLM/OCR papers:
ReadBench (arXiv:2505.19091), typographic perturbation work
(arXiv:2604.12371 / 2604.25102), and typography-gap work
(arXiv:2603.08497).
The main guidance for pixelpipe is: don't increase DPI; margin tweaks
are tiny; the real foundational experiment is a denser-but-readable
bitmap atlas (e.g. 5x11 -> 4x8/4x7) with exact retrieval tests before
shipping.
Production data (ocproxy 7d, Opus 4.7) shows pixelpipe HURTS short
conversations and helps long ones:
input.count <10 : +$0.17 (HURTS) over 14 rows
input.count 10-19: +$0.04 (HURTS) over 22 rows
input.count 20-39: -$0.56 (helps) over 29 rows
input.count 40+ : -$3.21 (helps) over 237 rows
Diagnosis: warm-cache share split.
bucket actual_warm_share baseline_warm_share
<10 60% 100%
10-19 84% 97%
20-39 88% 96%
40+ 92% 58%
For short conversations the un-rewritten path is already nearly fully
warm-cached. Rewriting the prefix bytes (slab compression / image
substitution / history collapse) gives the new prefix a different
cache key, so Anthropic charges cache_create (1.25×) on the new
prefix's first turn — destroying the prior warm cache. The
conversation often ends before the new cache amortizes.
Fix: add a 'priorWarmTokens' option to TransformOptions. When >0, the
break-even gate penalises the image side by
burn = priorWarmTokens × (CACHE_CREATE_RATE − CACHE_READ_RATE)
Per-turn gate eats the full burn; amortized gate spreads textLifetime
across N turns so longer horizons re-accept once savings outweigh burn.
Default 0 preserves byte-identical behavior for callers that don't
plumb it through (cold-start safe).
Tests: 3 new (NaN clamp, per-turn flip, horizon flip). 302 total pass.
count_tokens validates structural pairing and rejects truncated prefix
bodies that contain tool_use blocks whose tool_result was in the
dropped tail, with: 'messages.<N>: tool_use ids were found without
tool_result blocks'. That's been firing on ~15% of Opus 4.7 rows
(75/503 last 7d) and excluding them from baseline_probe_status='ok'.
Append a synthetic user message with minimal tool_result blocks for
any orphan ids. Adds ~5-10 tokens per orphan (within ~1% of truth)
and converts those rows from 'partial' to 'ok' so they contribute
to honest savings rollups.
Source: ocproxy daemon.log under OCPROXY_PIXELPIPE_DEBUG=1.
N=391 Opus 4.7 measured rows (last 7d) show real chars-per-token = 1.91
(avg_outgoing_text_chars=231,925 / avg_real_input_tokens=115,893). The
older 2.5 constant was calibrated on Opus 4.5/4.6 and over-estimated cpt
by ~30%, under-counting text-token cost and causing the break-even gate
to reject most history-collapse opportunities (283/391 = 72% of measured
ok rows came back history_reason='not_profitable').
Resize one render.test.ts fixture (6060 → 4848 chars) so the 'low cpt
unlocks a previously-rejected slab' assertion still exercises the
intended before/after edge under the new default.
Per-session AI-assistant memory is local-only — useful for context
continuity across sessions but never appropriate to ship to the public
repo. CLAUDE.md and HANDOFF.md (which were previously tracked) have
been purged from git history via filter-repo in the same operation
that introduced this gitignore rule.
If you cloned before this commit, your existing CLAUDE.md / HANDOFF.md
will continue to live in your working tree but won't show in git
status, which is the intended steady state.
Earlier commit a0f282f "corrected" Opus 4.7 rates downward to $2.50
input / $12.50 output based on a misread of the pricing source. Per
docs.claude.com/en/docs/about-claude/pricing (current), Opus 4.7 is:
Base input: $5 / MTok (same as Opus 4.5, 4.6)
Output: $25 / MTok
5m cache write: $6.25 / MTok
Cache hits/reads: $0.50 / MTok
Same dollar pricing as Opus 4.5 / 4.6 — what changed in 4.7 is the
tokenizer, not the per-token rate. The README now flags this so future
contributors don't assume hardcoded chars-per-token / image-token
constants tuned on earlier models are still safe on 4.7; only
count_tokens against the target model is honest.
Touches:
- src/dashboard.ts: ASSUMED_INPUT_USD_PER_MTOK back to 5.0, with a
long comment block citing the source and warning about tokenizer
differences. Updated card subtext on the headline accordingly.
- README.md: pricing table back to $5/$25/$6.25/$0.50, with a
callout that the tokenizer changed in 4.7.
The README headlined a '~76% fewer tokens' framing, a worked example
with a 'savings' column, and an Opus pricing table that used the old
Opus-4-base rates ($5/$25/MTok input/output) instead of Opus 4.7's
actual rates ($2.50/$12.50). Combined, it read like an aggregate
$-savings claim that the data does not yet support.
What's actually true and stays:
- Pixelpipe ships fewer input tokens on cold-miss requests, measured
by Anthropic's own count_tokens endpoint. The 173,783 -> 41,321
anecdote is one real request, not a session average.
- Image-tokens-vs-text is a real per-request encoding-density
observation.
What was misleading and is removed/softened:
- Opening tagline no longer claims '~76% fewer tokens, 100% quality'
as if it generalizes. Replaced with an experimental-status note.
- Worked-example 'savings' column renamed 'delta' and contextualized.
- 'How we report numbers' section added that explains the
baseline_probe_status gate, says we don't currently make an
aggregate $ saved claim, and warns the reader to treat any host
dashboard that surfaces $ saved without the gate as marketing.
- Pricing table corrected to Opus 4.7 rates.
When the cacheable-prefix count_tokens probe failed but the full-body
probe succeeded, the dashboard fell through to `baselineCacheableTokens
?? 0`. That silently treated the unproxied counterfactual as
cold-input-only — charging full $2.50/Mtok input rate on tokens that
demonstrably would have been cache-discounted at 10× cheaper. Net effect:
fabricated "$ saved" headline on every partial-probe row.
The fix:
1. New `TransformInfo.baselineProbeStatus: 'ok' | 'partial' | 'failed'`,
set by the proxy after awaiting both probes. 'ok' means either both
probes resolved OR the original body had no cache_control markers
(cacheable=0 is then exact, not estimated). 'partial' means the
full-body probe resolved but the cacheable-prefix probe didn't.
2. `tracker.ts` persists it as `baseline_probe_status` on the JSONL.
3. `dashboard.ts` gates its savings math on `status === 'ok'` (with a
back-compat path for legacy rows that pre-date the field). Partial /
failed rows now contribute to the request count but not to
baselineInputWeighted / actualInputWeighted / outputWeighted.
ocproxy's host-side dashboard already records baseline_probe_status in
its SQLite schema and is being updated in parallel to honor the same
gating semantic. See ocproxy commit fixing dashboard/src/lib/snapshot.ts
for the matching change.
The previous value was Opus 4.5/4.6 pricing. Opus 4.7 input is $2.50/Mtok,
output $12.50/Mtok. The dollar-saved headline was overstated by 2×.
OUTPUT_TOKEN_RATE (5×) and cache multipliers (1.25×/0.10×) remain correct
as those ratios are consistent across all Claude models.
Adds isCompressionProfitableAmortized() and a new
historyAmortizationHorizon option (default 1 = current per-turn
behaviour). When the host opts in to horizon ≥ 2, the history-collapse
gate evaluates expected lifetime cost over N future turns instead of
the cold per-turn comparison:
I × (CC + CR×(N-1)) < T × CR × N
where CC = 1.25 (cache_create), CR = 0.10 (cache_read)
At N=5 this accepts collapses with I/T < 0.30; at N=10 with I/T < 0.47.
The worst-case-for-image / best-case-for-text framing is on purpose —
we only collapse when the math wins under pessimistic warm-cache
assumptions, so a single-turn session can never pay a tax.
This is the 'try-then-decide-with-bounded-horizon' design from the
README's new 'unsolved part: multi-turn amortization' section. Picked
over JIT-style session state (rejected: introduces durable state),
always-collapse (rejected: dishonest per-request), and cache-bust-driven
(rejected: signal arrives one turn late).
- New option threaded through DEFAULTS, TransformOptions, PixelpipeOptions
- Both history call sites (early-exit helper + main success path) use the
amortized gate when horizon > 1, fall back to the cold gate otherwise
- 4 new unit tests covering fallback, accept-on-long-history, reject-on-
tiny-text, and the subset-relationship between cold and amortized gates
- 298 tests pass
The break-even gate in collapseHistory is per-turn — it answers 'is the
image cheaper than the text on this single request?' That question has
the wrong answer once Anthropic has cached the prior text-based prefix,
because text-at-10% beats image-at-40%-cold. But the prefix's lifetime
cost is what matters, not the per-turn cost, and lifetime cost favours
the image once the cache eventually expires.
This is the same shape of decision a JIT compiler, a DB optimiser, or
ZFS block compression makes. Documenting the analogues, the four
credible designs we considered (try-then-decide / session-state /
always-collapse / cache-bust-driven), and why we picked try-then-decide
as v1.
The Why-it's-hard section gets a new paragraph framing the asymmetric
cache invalidation. The history-compression section gets a new
'The unsolved part: multi-turn amortization' subsection with the full
design space and the decision criteria.
No code changes — this is design documentation for future contributors
so they don't reinvent the analysis.
teamchong