- ON 47823 / OFF 47824 with own PXPIPE_LOG files: operator session can
never pollute bench measurement (separation by construction)
- resume = skip existing patch files; clean stop on quota errors
- Docker grading documented as proxy-free (no model calls in containers)
- README: mention Pro pilot alongside Lite
Does the model actually read pixelpipe's render, or coast on memorized answers?
- novel random-number problems (un-memorizable): text 100% vs image 93% (-7pp real reading tax)
- GSM8K (in training data): 97% vs 96% -- inflated ~3pp by recall
- verbatim recall from a dense render: 0/15
README 'Benchmarks' section leads with the clean 93% and reports the failure
rows too (no cherry-pick). Harness in eval/gsm8k/: gen_novel.py, render_cfg.mjs
(pixelpipe's real renderTextToPngs), bench.py -- claude -p, exact-match grading.
Widen the model gate to /^claude-opus-4-(?:[7-9]|[1-9]\d)(?:-|$)/ (Opus 4.7
and newer; previously 4.6/4.7). Wire isPixelpipeSupportedModel into the proxy
boundary in proxy.ts — the proxy previously gated on economics only and
ignored the model, so it compressed 4.8 traffic despite the docs claiming a
4.6/4.7 scope. Unsupported models now pass through with reason
'unsupported_model'.
Update public-api tests to the 4.7+ spec (320/320 pass).
Correct the docs to live measurement: reverse the POSTMORTEM "dead" verdict
(pixelpipe is a lossy gist-compressor saving ~68% on real dense Claude Code
traffic; the verbatim 0/15 needle finding stands as a caveat, not the
verdict), rewrite README Status/Limitations, and add a correction pointer to
the eval README. Original POSTMORTEM body preserved below the correction.
BREAKING CHANGE: Opus 4.6 and older are no longer supported.
Verbatim-risk guard (skip imaging unique IDs/hashes/exact values) still pending.
Pixelpipe now always uses the full 1568px canvas (cols=313) and packs up
to 50,000 chars per image. The multi-column code path and the
shrink-to-content path are no longer used by transform.ts - both were
sacrificing token savings on dense content to make sparse content marginally
more readable, and the readability gain wasn't real (the renderer's
output was unreadable in either layout once the content was actually dense).
Key changes:
- READABLE_CHARS_PER_IMAGE: 6_000 -> 50_000
- DEFAULT_COLS: 100 -> 313 (full 1568px / 5px cell)
- shrinkColsToContent() is now a no-op (returns cols unchanged)
- MinToolResultChars / MinReminderChars default to 50k (was 6k)
- transform.ts always renders single-column at full canvas
Practical impact: a 6 KB tool_result that previously rendered as 4 narrow
508x488 pages (1324 image tokens, 86% of text cost) now renders as 1
full-canvas 1568x488 page (~331 image tokens, 22% of text cost) - 4x
more savings on the same content with no quality change.
Rebuilt dist/. Tests: 315/315 pass.
The reflow-inimage finding (commit bb9c231) made 5×8 viable, so
DEFAULT_CELL_W_BONUS / DEFAULT_CELL_H_BONUS reverted to 0 — but many
comments and the README still said 7×10 was production.
Touched:
- README.md — eval table, density bullet, sweep narrative,
cell-pitch section, measurement header,
Limitations section
- src/core/render.ts — RenderStyle docstring + cell-bonus docstrings
+ multicol intro comment
- src/core/transform.ts — TransformOptions.reflow docstring, DEFAULTS
comments (minReminderChars floor rationale,
maxImagesPerToolResult cap, reflow gate
rationale, anchor scaling formula, LINES_PER_IMAGE
worked example)
No code behavior changed — every constant still resolves through CELL_W /
CELL_H, and the eval harness still overrides the cell via cellWBonus /
cellHBonus when comparing variants. The 5×8 production path is what was
already shipped on 81550f6 (reflow + grayscale + L1/L2 harness) and
bb9c231 (reflow-inimage).
Co-render the OCR instruction into the same PNG as the content,
delimited by '===…===' bands, with the API system field dropped.
L1 OCR fidelity, Opus 4.7, 20 production blocks, 7×10 cell:
- baseline (text-only): 97.91% mean / 96.25% min
- reflow (separate system): 91.99% mean / 82.59% min (-5.93pp)
- reflow-inimage: 98.95% mean / 96.42% min (+1.04pp)
reflow-inimage wins on all 20/20 blocks vs reflow, and beats the
text-only baseline on 17/20 blocks. Three blocks hit 100%. The
-5.93pp reflow regression that the cell-pitch sweep partially
recovered disappears entirely when the instruction is co-rendered.
Mechanism: when system carries the instruction and the image
carries the content, the model does cross-modal binding to figure
out what the image is for. Co-rendering reduces it to a
single-modal task with an unambiguous parse rule.
Files:
- eval/eval-l1-ocr.mjs: add reflow-inimage variant + prompt
- README.md: new section before history compression
- eval/EXPERIMENT_LOG.md: attempt #2 writeup
- eval/results/{l1-report.md, l1-results.json}: regenerated
Capture the practical conclusions from recent VLM/OCR papers:
ReadBench (arXiv:2505.19091), typographic perturbation work
(arXiv:2604.12371 / 2604.25102), and typography-gap work
(arXiv:2603.08497).
The main guidance for pixelpipe is: don't increase DPI; margin tweaks
are tiny; the real foundational experiment is a denser-but-readable
bitmap atlas (e.g. 5x11 -> 4x8/4x7) with exact retrieval tests before
shipping.
Earlier commit a0f282f "corrected" Opus 4.7 rates downward to $2.50
input / $12.50 output based on a misread of the pricing source. Per
docs.claude.com/en/docs/about-claude/pricing (current), Opus 4.7 is:
Base input: $5 / MTok (same as Opus 4.5, 4.6)
Output: $25 / MTok
5m cache write: $6.25 / MTok
Cache hits/reads: $0.50 / MTok
Same dollar pricing as Opus 4.5 / 4.6 — what changed in 4.7 is the
tokenizer, not the per-token rate. The README now flags this so future
contributors don't assume hardcoded chars-per-token / image-token
constants tuned on earlier models are still safe on 4.7; only
count_tokens against the target model is honest.
Touches:
- src/dashboard.ts: ASSUMED_INPUT_USD_PER_MTOK back to 5.0, with a
long comment block citing the source and warning about tokenizer
differences. Updated card subtext on the headline accordingly.
- README.md: pricing table back to $5/$25/$6.25/$0.50, with a
callout that the tokenizer changed in 4.7.
The README headlined a '~76% fewer tokens' framing, a worked example
with a 'savings' column, and an Opus pricing table that used the old
Opus-4-base rates ($5/$25/MTok input/output) instead of Opus 4.7's
actual rates ($2.50/$12.50). Combined, it read like an aggregate
$-savings claim that the data does not yet support.
What's actually true and stays:
- Pixelpipe ships fewer input tokens on cold-miss requests, measured
by Anthropic's own count_tokens endpoint. The 173,783 -> 41,321
anecdote is one real request, not a session average.
- Image-tokens-vs-text is a real per-request encoding-density
observation.
What was misleading and is removed/softened:
- Opening tagline no longer claims '~76% fewer tokens, 100% quality'
as if it generalizes. Replaced with an experimental-status note.
- Worked-example 'savings' column renamed 'delta' and contextualized.
- 'How we report numbers' section added that explains the
baseline_probe_status gate, says we don't currently make an
aggregate $ saved claim, and warns the reader to treat any host
dashboard that surfaces $ saved without the gate as marketing.
- Pricing table corrected to Opus 4.7 rates.
The break-even gate in collapseHistory is per-turn — it answers 'is the
image cheaper than the text on this single request?' That question has
the wrong answer once Anthropic has cached the prior text-based prefix,
because text-at-10% beats image-at-40%-cold. But the prefix's lifetime
cost is what matters, not the per-turn cost, and lifetime cost favours
the image once the cache eventually expires.
This is the same shape of decision a JIT compiler, a DB optimiser, or
ZFS block compression makes. Documenting the analogues, the four
credible designs we considered (try-then-decide / session-state /
always-collapse / cache-bust-driven), and why we picked try-then-decide
as v1.
The Why-it's-hard section gets a new paragraph framing the asymmetric
cache invalidation. The history-compression section gets a new
'The unsolved part: multi-turn amortization' subsection with the full
design space and the decision criteria.
No code changes — this is design documentation for future contributors
so they don't reinvent the analysis.
- Headline: "pixels instead of tokens" → "pixels instead of text" (images
bill as tokens too — the honest framing is pixels vs text)
- Removed the redundant "Why this works in 2026" blockquote since the
Opus 4.7 explanation already lives in the intro paragraph
The intro now leads with the hypothesis (pixels pack more semantic info
per unit of context than serialized text) and treats the token reduction
as measurable evidence rather than the headline.
Adds a "What this is NOT" section to address the ToS / billing-loophole
read directly:
- Not ToS evasion (uses the documented vision API as documented)
- Not a billing loophole (savings hold even if Anthropic re-prices)
- Not cost arbitrage (cost is a side effect of density)
Same numbers, same code, different stance: defensive about the framing
("I'm not paying for churn I didn't cause") rather than offensive about
the pricing.
Pre-4.7 vision couldn't reliably OCR dense monospace glyphs - errors
would corrupt the prompt before the model read it. Opus 4.7 bumped
the long-edge image cap 1568->2576px (3.3x pixels) and document-OCR
benchmarks (DocVQA 87->94%, ChartQA 80->88%) per Anthropic's release
notes.
Pixelpipe still renders at 1568x1568 - this is a model fidelity
change, not a renderer change. But it's why this approach works now
when it didn't a year ago.
Cites anthropic.com/news/claude-opus-4-7.
Proxy targets Opus 4.7. Input pricing has been flat across Opus 4.5/4.6/4.7
($5/$25 per MTok), so the historical Opus 4 column ($15/$75) was framing
noise unrelated to actual cost.
~80-line section covering the 4-breakpoint cache cliff that history
collapse addresses, the closed-prefix walk, the 4 honesty gates, and
today's HISTORY_CHARS_PER_TOKEN=2.5 fix wired in b563751.
Links to the live data point in events.jsonl (12:30:01 event,
collapsed_turns=175, collapsed_chars=180,684) and the relevant
transform.ts / core/history.ts call sites.
teamchong