A proxy for Claude Code that intercepts POST /v1/messages and renders the bulky static inputs (system prompt + tool docs + closed-prefix history) as grayscale PNGs, letting Opus 4.7's vision stack OCR them on the way in. The hypothesis: pixels are a denser encoding for the same information.

Status: research / experimental. Pixelpipe demonstrably ships fewer input tokens on cold-miss requests (measured by Anthropic's own count_tokens endpoint). Whether that translates into a real end-to-end dollar saving on a multi-turn session depends on cache behavior, output / thinking tokens, and break-even math we are still measuring. We don't currently make a "$ saved" claim.

Opus 4.7 only. Pre-4.7 vision wasn't accurate enough on dense monospace glyphs — OCR errors would corrupt the prompt before the model read it. Opus 4.7's vision stack (released 2026-04-16) bumps the long-edge image cap from 1568 px to 2576 px (3.3× more pixels) and reports document-OCR benchmark gains large enough (DocVQA 87→94%, ChartQA 80→88%) to make this trade safe. Pixelpipe still renders at 1568×1568 — the model's OCR fidelity is what changed, not the renderer.

The inputs we touch today:

system field — Claude Code's base system prompt + CLAUDE.md project instructions + every loaded skill's SKILL.md + agent definitions. Identical every turn, ~tens of KB on its own.
tools field — every built-in tool schema (Bash, Read, Edit, Grep, Task, …) plus every MCP server's tool definitions. Each MCP server you wire up (Gmail, Calendar, Drive, custom servers) adds its full tool list here. This is usually the biggest single bucket.
closed-prefix tool_use / tool_result history — turns past the 4-breakpoint cache cliff that can't cache anymore, collapsed into one synthetic prepended user message + PNG.
large tool_result blocks in the live tail (long file reads, big bash outputs, MCP responses) that pass a per-block break-even check.
large <system-reminder> blocks inside user messages — Claude Code injects these for things like task-tracker state, file-state hints, and skill discovery; they grow with session length.

The current implementation renders flat text into monospace PNGs, which is the cheapest shape to validate the hypothesis on. The same idea generalizes to richer encodings — HTML system prompts with semantic hierarchy, tool docs as call graphs, file trees as tree renderings, conversation history as designed timelines, tables as actual tables. Concatenating text is how 2024-era prompt engineering works; designing a visual surface is what context-as-UI looks like.

On one measured cold-miss request in events.jsonl, Anthropic's count_tokens reported 173,783 input tokens for the unproxied body and 41,321 cache_create tokens for the proxied body — a ~76% reduction in tokens shipped on that single request. That's an encoding-density observation about one request, not an aggregate $-savings claim. See "How we report numbers" below for the difference.

What this is NOT

Not ToS evasion. Pixelpipe uses the Anthropic vision API exactly as documented. Image tokens are billed at the documented input rate. No reverse-engineering, no rate-limit circumvention, no API abuse — every encoded byte traverses Anthropic's stack under the same per-request budgets and content policies as a plain-text request.
Not a billing loophole. The savings come from genuinely shipping fewer tokens, measured by Anthropic's own count_tokens endpoint before and after the encoding change. If Anthropic re-prices images tomorrow, the encoding-density argument still stands — text is just no longer the cheaper modality.
Not a cost-arbitrage tool. The token reduction is the measurable proof that pixels pack more semantic info than serialized text. The actual claim — and the reason this repo exists — is that the LLM context window deserves to be designed like a UI, not concatenated like a log file. Cost is a side effect of density.

Runs on Node 18+ and Cloudflare Workers from the same source.

                                  ┌─ original ────────────────────┐
                                  │ ~68K input tok                │
Claude Code  ──►  pixelpipe  ──►  │  (system + tools as text)     │  ──►  Anthropic
                       │          └───────────────────────────────┘
                       └──────►   ┌─ via proxy ───────────────────┐
                                  │ ~3.5K input tok               │
                                  │  (system + tools as PNG +     │
                                  │   prompt-cache breakpoint)    │
                                  └───────────────────────────────┘
                                          ↓ Anthropic vision OCR
                                          100% reasoning quality retained

Why it works (the math)

The proxy intercepts POST /v1/messages, pulls the system prompt + tool documentation out of the JSON body, renders it into one or more grayscale PNGs using a build-time-generated hybrid glyph atlas: Spleen 5×8 for printable ASCII/code glyphs, with GNU Unifont 8px fallback for ~35k BMP codepoints by default — Latin extended, Cyrillic, Greek, CJK, Hiragana, Katakana, Hangul, Hebrew, Arabic, math symbols, box drawing, decorative symbols. It substitutes those PNGs back in as image content blocks with an ephemeral cache_control breakpoint.

Three independent derivations, each anchored on a number you can verify against the source.

Step 1 — image → tokens

Anthropic bills images by area (Vision docs):

image_tokens ≈ (width × height) / 750

Pixelpipe renders at a 5×8-px cell (see Why it's hard → Image density); a single-column cols=100 PNG is 508×1568, so the textbook estimate is:

508 × 1568 / 750 ≈ 1,062 tokens/image

Real count_tokens probes bill far above that textbook lower bound. Pixelpipe anchors on a measured 2,500 tokens for a single-col PNG at the 5×8 production cell (a 508-px-wide canvas) and scales linearly by canvas width: at multiCol=2 the same height doubles the width, working out to ≈ 5,500 tokens/image (the +10 % margin in effectiveTokensPerImage(numCols) absorbs extrapolation noise). Those are the TOKENS_PER_IMAGE_SINGLE_COL / effectiveTokensPerImage(numCols) constants in src/core/transform.ts, derived from CELL_W/CELL_H in src/core/render.ts so the gate tracks the renderer's real geometry.

Step 2 — text → tokens

The "English prose ≈ 4 chars/token" rule from Anthropic's pricing docs does not survive contact with real Claude Code traffic. Across N=391 production count_tokens probes on Opus 4.7 /v1/messages bodies:

avg outgoing text chars  231,925
avg real input tokens    115,893
observed mean            1.91 chars/token

Real bodies are JSON-dense — tool definitions, schemas, structured CLAUDE.md slabs, tool_result blocks — which tokenize 2-4× denser than prose. The gate isCompressionProfitable() uses SLAB_CHARS_PER_TOKEN = 2.0 at the slab call site (slightly conservative versus the observed 1.91 cpt), so it only compresses when the text actually costs more tokens than the image will. At the textbook 4 ch/tok the gate silently rejects every realistic slab as not_profitable — that bug is what motivates the constant.

Step 3 — tokens → $

Rates from Anthropic's pricing page for Opus 4.7 (input pricing has been flat across 4.5/4.6/4.7). Image tokens are billed at the input rate.

line item	rate
input	$5.00 / MTok
output	$25.00 / MTok
cache_create (5 min)	$6.25 / MTok
cache_read	$0.50 / MTok

Opus 4.7 uses a different tokenizer than 4.5 / 4.6 (per Anthropic's pricing page). The same input string does not produce the same token count across models, so any hardcoded image-token / chars-per-token constants in pixelpipe were tuned on an earlier tokenizer and may be biased on 4.7. The break-even gate's only honest oracle is count_tokens against the actual target model.

Worked example — one real cold-miss event

From events.jsonl, 2026-05-20T12:30:01 (a fresh session, 161,101-char system slab + 37 images-worth of accumulated history):

orig_chars             161,101    system + tool docs slab
image_count                 37
baseline_tokens        173,783    count_tokens probe of the unproxied
                                  body — what Anthropic would have
                                  billed without the proxy
cache_create_tokens     41,321    what actually got billed via pixelpipe
cache_read_tokens            0    cold miss

Byte reduction on the cold miss: 76% (173,783 → 41,321 tokens). Same event, run later in the session (11:53:06, warm hit on the cached PNGs):

baseline_tokens        168,707
cache_create_tokens        111    only the per-turn dynamic delta
cache_read_tokens      140,786    paid at 0.1× the input rate

metric	original	via proxy	delta
Cold input tokens (per call)	~174k	~41k	~76% fewer
Cache-warm input tokens	~169k	~141k	~17% fewer
Per-image OCR quality vs txt	-	-	~99.5%

These are per-request token-count deltas, not a session-level cost claim. A real session interleaves cold-miss and cache-warm calls, includes output / thinking tokens we don't touch, and depends on cache TTL and Claude Code's usage shape — none of which a single-request delta captures.

How we report numbers

Pixelpipe instruments every proxied request with two free count_tokens probes on the original uncompressed body — one full, one truncated at the last cache marker — and persists them alongside Anthropic's billing usage block. The bundled dashboard uses those to show token deltas per request and aggregate token counts, and refuses to display a "$ saved" headline unless both probes succeeded and the host has wired in pricing. On real traffic to date, the honest aggregate is closer to break-even than the cold-miss number above suggests; the value proposition is still under measurement.

If you see a "$ saved" number coming out of a host integration that doesn't expose this gating, treat it as marketing, not measurement.

Why it's hard (the parts that bite)

Most of the engineering in this repo is not "render text to PNG." That part is a build-time atlas and a Uint8Array blit. The hard parts are all about what is safe to compress, when, and at what cost.

Prompt caching changes the question. Anthropic's cache writes cost 1.25× normal input (cache_create), reads cost 0.1× (cache_read), the TTL is 5 minutes, you get 4 breakpoints per request, and any change to prefixed content invalidates everything downstream. A naive "compress everything" proxy would lose money on warm requests where 90% of the slab was already cached at 10% billing — the image still costs its full ~5,500 tokens at cache_create price the first time it's seen. The proxy stays a win because (a) the image's cache_create is still 95%+ cheaper than the text's cache_create, and (b) Claude Code sessions are bursty: 5-15 min coding bursts separated by lunch / meetings / context-switching hit cache expiry constantly. Every ~5 min idle = a fresh cold miss = the 1.25× tax re-paid on the full uncompressed slab. Cache expiry argues for compression, not against it — the tax stays the same, the base shrinks.

The static slab is mostly free; the real headroom is dynamic. Once the system prompt + tool definitions are cached, they bill at 10% on every warm turn. That's where Anthropic's "just use the cache" guidance ends. But Claude Code's tool_result blocks change every turn, never cache, and pay full freight every single turn — a 30k tool_result × 10 turns = 300k uncached tokens billed at 100%, bigger than the slab fix. History compression (Variant C) addresses the other side: long sessions push older turns out of the 4-breakpoint cache budget, so a 50-turn session with 30 turns of tool_results pays 600k tokens of uncached, repeat-billed text on every turn. Collapsing those into one synthetic prepended user message + PNG is the same shape of fix as the slab.

Assistant outputs and the model's thinking cannot be image-encoded. This is a hard architectural constraint, not a tuning choice. The Anthropic Messages API only accepts image content blocks inside user messages — assistant turns are text-only by contract. And even if the API allowed it, the proxy never sees an assistant token until after the model has generated it; you can't render what hasn't been emitted yet. The same applies to extended-thinking blocks: they're produced by the model, billed as output, and round-tripped on subsequent turns as opaque assistant content. Everything pixelpipe compresses is input-side, host-supplied, and known before the call. That's why the two compression paths are (1) the static slab — system prompt + tool docs that Claude Code injects identically every turn — and (2) closed prefix history — user/tool_result turns that have already happened and will never change. Anything the model wrote, or will write, is off the table.

The break-even gate has to be honest about real text shape. A 161k production slab was being silently rejected as not_profitable for weeks because the gate estimated text at the textbook 4 chars/token when the real density was 1.17. The gate's job is "compress if and only if doing so saves tokens" — if the constant is too low we miss profitable compressions, if it's too high we accept money-losers. The fix wasn't a flag; it was wiring a chars_per_token value derived from a parallel /v1/messages/count_tokens probe into the call site that owns the decision. The same fix applied at the history call site unlocks historyReason: "collapsed" for long sessions.

Image density / font size: more DPI is not the answer. Recent VLM/OCR work points the tuning direction pretty clearly. ReadBench (arXiv:2505.19091) renders text-only benchmarks as images and reports that text resolution has negligible effects once the text is readable, while performance drops sharply on longer multi-page visual contexts. Typographic attack studies over GPT-4o, Claude Sonnet 4.5, Mistral, and Qwen (arXiv:2604.12371, arXiv:2604.25102) find a real small-font cliff: very small fonts (~6 px) become ineffective, while mid-range font sizes are read reliably. A typography-gap study (arXiv:2603.08497) reinforces the practical lesson: VLMs are much better at reading what text says than recognizing font family/style. For pixelpipe this means:

Do not increase DPI / pixel dimensions to improve savings. More pixels usually means more image tokens.
Tiny margins are fine, but shrinking PAD_X/PAD_Y from 4 px to 2 px is only a ~1% win; it is not the main lever.
Density is the cell pitch, not a letter-spacing knob: the production render cell is 5×8 px — the bare Spleen 5×8 glyph bitmap with DEFAULT_CELL_W_BONUS / DEFAULT_CELL_H_BONUS both 0. The eval/-only cellWBonus / cellHBonus overrides sweep other sizes for A/B testing.
We ship Spleen 5×8 glyphs for ASCII/code plus Unifont 8px fallback, after 4×8 proved too brittle in exact code-reading tests.
Gutter/padding changes are secondary; the gutter is an OCR-ordering cue for multi-column layouts, so removing it can save pixels while silently causing row-interleaved reads.

Measured (eval/): the legibility story is a two-step. An L1 OCR-fidelity harness renders real Claude Code transcript blocks, sends them through Anthropic's vision stack, and scores the transcription character-for-character against the source.

Step one — diagnose the failure. At a bare 5×8 cell with naïve newline-rendered text, OCR was content-dependent: ~97 % on sparse text (real newlines, lots of whitespace), but ~81 % mean on dense text, with individual blocks below 25 %. This is content-agnostic at the encoder: it sees pixels, not newlines, and a reflow-packed block and a zero-newline minified JSON produce the same dense pixels and fail the same way. A cell-pitch sweep on Opus 4.7 (20 blocks) found that spacing out the glyph to 7×10 recovered accuracy:

Render cell	Mean accuracy	Δ vs baseline	Notes
5×8 (bare)	81 %	−16 pp	dense-text failure
6×9	90 %	−7 pp
7×9 / 6×10	~93 %	−4 pp
7×10	96.5 %	−0.8 pp	spaced 5×8 bitmap
8×10	93 %	−5 pp	over-spacing dilutes

7×10 was the densest cell where OCR held with the naïve renderer. Spacing a 5×8 bitmap was paying for accuracy with ~75 % more pixels per character — workable, but it narrowed the token-economics margin to break-even on average text.

Step two — rehabilitate the cell. Three later changes together moved the failure mode off pixel-pitch entirely and brought 5×8 back as the shipping cell:

Packed reflow (reflow: true, wrapLines) — strips trailing whitespace and wraps every source line onto its own visual row, marked with a ↵ (U+21B5) sentinel where the source had a real newline. Net loss of ~−1 pp of accuracy at the cell level vs. baseline-newlines, but it removes the line-end dead margin (the ~29 % glyph fill that made 5×8 expensive without making it more legible).
Grayscale (anti-aliased) atlas (atlas-gray.ts, aa: true) — the renderer can blit either the 1-bit Spleen atlas or an 8-bit anti-aliased version of the same font. AA edges survive the vision encoder's downsample at a smaller cell, raising the floor below 7×10.
In-image instruction band (reflow-inimage variant) — the OCR-style "transcribe this verbatim" instruction rides inside the PNG above a delimiter band, instead of being shipped in the API system field. Eliminates the cross-modal binding step (see Eval finding: render the instruction inside the image below).

The combined result, measured on 20 production blocks at the 5×8 cell:

Variant	Mean Acc	Δ vs baseline
baseline (text-only, no image)	97.91 %	—
reflow (image + separate `system`)	91.99 %	−5.93 pp
reflow-inimage (instruction inside the PNG)	98.95 %	+1.04 pp

reflow-inimage at the 5×8 cell beats the text-only baseline on 17 of 20 blocks. So production now runs:

DEFAULT_CELL_W_BONUS = 0, DEFAULT_CELL_H_BONUS = 0 (bare 5×8 cell, CELL_W = ATLAS_CELL_W = 5, CELL_H = ATLAS_CELL_H = 8)
reflow: true in DEFAULTS
the in-image instruction band prepended by transform.ts

The break-even gate is recalibrated to the 5×8 production geometry: TOKENS_PER_IMAGE_SINGLE_COL and LINES_PER_IMAGE both derive from CELL_W / CELL_H in render.ts (508-px canvas, 2,500 tokens/image single-col, 5,500 at multiCol=2, 195 visual rows per image). The 7×10 cell remains available via the eval/-only cellWBonus / cellHBonus overrides for A/B comparison; it is no longer the production default.

Multi-column packing has an OCR cliff. Two columns side-by-side double the per-image text capacity, but the renderer must guarantee Anthropic's vision stack reads column 1 fully top-to-bottom before column 2. Layouts with line lengths near 1568 px and a weak column divider can produce row-interleaved OCR output. The renderer adds a light-gray gutter divider, a per-image break-even check specifically for multiCol=2 (image cost scales with numCols plus a 10 % extrapolation margin — effectiveTokensPerImage(2) ≈ 7,665), and a global multiCol: 2 default that can be overridden to 1 if OCR ordering ever turns out wrong on a specific deployment.

Cache prefix invalidation is asymmetric in time. Anthropic matches prompt cache by byte prefix, so any change to the request changes what caches downstream of that change. The first turn we replace a chunk of message history with a PNG, the prior text-based prefix becomes wasted bytes from Anthropic's POV — cache flushes from the change-point onward and we pay cache_create (1.25×) on the new image. Subsequent turns with the same image bytes get cache_read (0.1×) on the image. So history compression is multi-turn economics: a single-turn break-even gate that asks "is image_tokens < text_tokens cold?" always says no once Anthropic has already cached the text — text-at-10% beats image-at-40%. But that reasoning ignores that the text cache will expire / get evicted / hit the 4-breakpoint cliff anyway, and when it does, the next cold call pays full freight on the giant text prefix. The honest framing is expected lifetime cost of the prefix in this session vs. expected lifetime cost if we collapse now. Neither number is locally observable. The same shape of problem shows up in database indexes (cost of building amortizes over future queries), JIT compilation (interpret first, compile what proves hot), and ZFS block compression (compress speculatively, keep only if it shrinks ≥ a threshold). Pixelpipe today uses a per-turn chars/token gate — the JIT analogue is "always interpret." We have data on individual turns but no session-scoped amortization model yet, which is why history collapse declines as not_profitable on warm Codex traffic even when the long-run answer would be "collapse and let cache_read recoup it." The next iteration is the ZFS-style try-then-decide path: render, count rendered tokens against a parallel count_tokens probe on the pre-collapse text, commit the collapse only if the difference exceeds a multi-turn break-even (e.g. image_tokens × (1+0.1·N) < text_tokens × (1+0.1·N) for N ≥ a configured amortization horizon). Honest, local, deterministic, no session-state required — at the cost of ~30 ms of wasted render CPU on turns we end up discarding. Worth it to stop guessing.

**Telemetry is the only honest oracle. Every constant in the gate — SLAB_CHARS_PER_TOKEN, HISTORY_CHARS_PER_TOKEN, LINES_PER_IMAGE, TOKENS_PER_IMAGE_SINGLE_COL, effectiveTokensPerImage(numCols) — has a comment pointing at the production probe that grounded it, with date and sample size. The proxy logs every request to events.jsonl with baseline_tokens (from a parallel cold count_tokens call), cache_create_tokens, cache_read_tokens, orig_chars, image_count, image_pixels, outgoing_text_chars, and (when history fires) collapsed_turns, collapsed_chars, collapsed_images. The dashboard at / regresses total_tokens = α·outgoingTextChars + β·imagePixels on every cold-miss event to keep the per-image cost estimate honest as the model and atlas evolve.

Proving a compression is safe needs the real workload — not SWE-bench. Pixelpipe has one speculative lever, reflow, that packs text denser by replacing newlines with a ↵ (U+21B5) glyph so the renderer fits ~970 chars/image instead of ~470. Whether that lever is safe to pull depends entirely on one question: can the model still read the text it is given? SWE-bench is the wrong instrument for that question. SWE-bench instances are essentially single-shot bug-fix tasks — they never accumulate the hundreds of turns of tool_result history that reflow actually operates on, and a SWE-bench score moves for a hundred reasons unrelated to whether one ↵ glyph was resolved correctly. The workload that genuinely stresses reflow is our own multi-turn Claude Code sessions, and those already exist on disk: ~/.claude/projects/ holds real transcripts and events.jsonl holds ~7,900 real proxied requests. So the eval harness (eval/) replays that real corpus instead of a synthetic benchmark, in two tiers. L1 — OCR fidelity: render each text block both ways (baseline newlines vs. reflow ↵), ask the model to transcribe it verbatim, and score against ground truth by character-level Levenshtein distance. Cheap, deterministic, and it isolates the one variable. L2 — session replay: rebuild whole sessions with the collapsed-history PNG, ask a real question, and have a separate judge model score whether the reflowed answer is equivalent to the baseline answer. The shipping gate requires L1 accuracy delta ≥ −2pp, L1 macro accuracy ≥ 95%, L2 judge score ≥ 0.80, and L2 pass rate ≥ 80%. All four must hold.

The eval must run against the production model, and it found the real bottleneck. Run at a naïve 5×8 cell on Opus 4.7, reflow dropped L1 character accuracy from 97.67 % → 80.59 % (−17 pp) while saving 0.0 % image area, and L2 session replay scored a 49.5 % mean judge score. A Sonnet 4.5 control produced near-identical numbers (−17.09 pp at L1) — and that was the key finding: the failure was not model vision capability (Opus reads the baseline render at 97.67 %) and not ↵ comprehension (enlarging and recoloring the marker in dedicated variants changed nothing). It was raw pixel density at the cell scale. The cell-pitch sweep above followed directly from that result, and 7×10 was the first cell where dense reflowed text held at ~96.5 % L1. The follow-on reflow-inimage change — co-rendering the OCR instruction band inside the PNG above a delimiter — brought the 5×8 cell back to 98.95 % L1, slightly above the text-only baseline, which is why production reverted to the bare 5×8 cell paired with reflow: true and the in-image instruction. The L2 session-replay re-run on the shipped 5×8 + reflow-inimage config is the remaining confirmation — the harness makes it one node eval/run-eval.mjs --level all --confirm --model <production-model> away from a verdict instead of a guess.

Eval finding: render the instruction inside the image

A late finding worth surfacing on its own, because it inverts the intuition about where to put the OCR-style prompt that tells the model what to do with a pixelpipe image.

The setup. When pixelpipe ships text as PNG, something still has to tell the model "decode the pixels back to text." Two places that instruction can live:

Separate system field — the natural API choice. The image is the user content; the system field carries the rendering contract.
Co-rendered into the same image — the instruction band sits above a delimiter; the content sits below; the model sees one PNG.

The measurement (L1 OCR fidelity, Opus 4.7, 20 production blocks, 5×8 cell, packed reflow ON):

Variant	Mean Acc	Δ vs baseline	Worst block
`baseline` (text-only, no image)	97.91%	—	96.3%
`reflow` (image + separate `system`)	91.99%	−5.93pp	82.6%
`reflow-inimage` (image carries the instruction)	98.95%	+1.04pp	96.4%

The in-image variant wins on every one of 20 blocks vs the separate- system reflow, and beats the text-only baseline on 17 of 20 blocks. The −5.93pp reflow regression that the cell-pitch sweep partially clawed back disappears entirely when the instruction is co-rendered.

Why it works. When the instruction lives in system and the text lives in user.content[].image, the model has to do cross-modal binding: "the system field says transcribe, and separately there's an image — what is it?" The vision encoder reads the image once; the instruction arrives through the text path; the two have to be reconciled at decode time, and the image gets treated as ambiguous input (sample? example? content to read?).

When the instruction is co-rendered with the content above a clear delimiter, it's a single-modal task: "read this image, follow the section above the delimiter, output the section below." The vision encoder reads the instruction at the same fidelity as the content; the parsing rule is unambiguous; no cross-modal coordination is required.

What it means for production. src/core/transform.ts already prepends a synthetic user message containing the system+tools image, so the architecture is half-there — the system prompt is not sitting in the real system field at request time. The remaining lever, which this eval quantifies, is whether to render a small instruction band into that image. The L1 result says yes; the L2 session-replay rerun (judge-scored comprehension, not character accuracy) is the production confirmation.

Repro: node eval/eval-L1-ocr.mjs --confirm --model opus --variants baseline,reflow,reflow-inimage --max-blocks 20

How history compression works

This is Variant C in src/core/transform.ts (the collapseHistory path). It addresses a different cache cliff than the static slab.

The problem it solves

Claude Code uses 4 prompt-cache breakpoints. The static slab (system + tools + CLAUDE.md) holds one. The remaining 3 live the conversation tail. In a long session:

[system slab][turn 1][turn 2]...[turn 50][turn 51 live]
              ↑                    ↑
              cache breakpoint     cache breakpoint
              from 40 turns ago    on current turn

When the gap between the oldest cached breakpoint and the live tail exceeds the 4-breakpoint cache budget, every new turn page-faults the historical prefix and Anthropic re-bills the whole tool_result river at 1.25× cache_create. A 50-turn session with 30 turns of fat tool_result blocks (file reads, bash outputs, MCP responses) can pay ~600k tokens of uncached, repeat-billed text every turn.

The collapse

collapseHistory() walks from oldest message forward looking for the closed prefix — the longest run of turns that's guaranteed never to change again (no open tool_use waiting for a result, ends on a clean user/assistant boundary). It serialises that whole run into one giant block of text and feeds it to the same renderTextToPngs pipeline the slab uses.

Output looks like:

[system slab]
[synthetic user message 1: "[Earlier in this conversation:]"
  └ image block: PNG of 175 turns serialised as JSON
  └ text block: "[End of earlier context.]"]
[turn 48 (last 2 turns kept verbatim in "Live tail")]
[turn 49]
[turn 50 live]

keepTail: 4 is the default — the 4 most recent turns stay as native messages so the model still has structured access to recent tool_use/tool_result pairs.

Why it's tricky

Three honesty gates have to all clear:

historyReason: 'no_history' — only one message, nothing to collapse.
'prefix_too_short' — the closed prefix is under minCollapsePrefix: 10 turns.
'no_closed_prefix' — every prefix ends on an open tool_use (mid-call). Common in single-turn smoke tests.
'not_profitable' — the gate (isCompressionProfitable) decided the text was so sparse the image cost would exceed the text cost. This is the one that was firing wrong before today's e8545a9 commit.
'collapsed' — actually fired.

Today's fix (the `42ef4c5` commit)

History was using charsPerToken: 4 (Anthropic's English-prose default). Real chat-shaped JSONL ([{role: 'user', content: [{type: 'tool_result', content: '...'}]}]) tokenizes denser than prose. The N=10 rejected history events in events.jsonl had real cpt 1.08–1.10 — every one of them was a profitable compression the gate dropped on the floor.

Fixed by wiring HISTORY_CHARS_PER_TOKEN = 2.0 at the call site (same shape as the slab's SLAB_CHARS_PER_TOKEN = 2.0 Opus-4.7 calibration). Live data after restart shows it firing: the 12:30:01 event in events.jsonl has historyReason: 'collapsed', collapsed_turns: 175, collapsed_chars: 180,684.

The unsolved part: multi-turn amortization

The break-even gate above (isCompressionProfitable) is per-turn — it asks "on this single request, is the image cheaper than the text?" That question has a clean answer when both sides are cold (no prompt-cache hits), but it has the wrong answer when Anthropic has already cached the prior text-based prefix:

state	text cost	image cost	per-turn winner
cold (turn 1, fresh session)	1.00×	0.40× of text-token-count	image
warm with cached text prefix	0.10×	0.40× cold → 0.10× after	text
cache expired (5-min idle / 4-bp eviction)	1.00× again	re-cached image hits at 0.10×	image

A pure per-turn gate happily collapses on cold and correctly refuses on warm — but it can't see that a warm session will eventually go cold again, and on that cold turn the giant text prefix pays full freight while the image prefix pays once and then rides cache for the rest of the session.

This is the same shape of decision a JIT compiler makes (interpret first turns, compile what proves hot), a DB optimiser makes (build the index if N future queries amortize the scan), or ZFS makes (compress the block, keep the compressed form if it shrinks by ≥ 12.5%). None of them rely on knowing N exactly. They all rely on a bounded amortization horizon baked into the gate: assume N=K turns, decide once, eat the loss if K turned out to be smaller.

Pixelpipe today is the "always interpret" mode. The design space for fixing it has four credible options. Documenting all of them — including the ones we rejected — so the next contributor doesn't reinvent the analysis.

Option A — Try-then-decide (ZFS block-compression analogue)

Render the closed-prefix history to PNG(s) speculatively, count actual image tokens via a parallel count_tokens probe, compare against text tokens for the same prefix evaluated at a fixed amortization horizon (e.g. N=5 future turns, accept iff image_tokens × (1 + 0.1·(N-1)) < text_tokens × (1 + 0.1·(N-1))). Commit the image only if it wins; discard the render otherwise — ~30 ms of wasted CPU on the daemon, no token cost.

Pro: local, deterministic, no session state, no future-knowledge assumption. Honest about the horizon and accepts bounded waste on misjudgments. Composes with all the other gates we already have.
Con: one extra count_tokens round-trip per request that's considering collapse. Wasted render CPU on rejects (~30 ms each at current image counts).
Status: chosen path. Specced; not yet implemented.

Option B — Session-state aware (JIT tiered-compilation analogue)

Derive a session id from request shape (e.g. hash of the first user message + system slab), track per-session {turn_count, cache_state, last_render_decision} in the host, leave history as text for turns 1..K, collapse once K is exceeded. ocproxy already has cache_session_hash which can stand in as the session id.

Pro: matches the JIT pattern exactly — interpret first, compile what proves hot. Avoids speculative render cost on short-lived sessions. Cleanest economics on long sessions.
Con: introduces durable state. State means schema, eviction, migration. State means "why is pixelpipe behaving differently on identical inputs?" debugging. Cross-process / cross-host coordination if the daemon restarts or fans out. The honesty cost is high relative to the marginal win over Option A.
Status: rejected for v1 of the fix. Revisit if Option A leaves measurable money on the table after a few weeks of data.

Option C — Always collapse, trust the law of large numbers (CDN analogue)

Drop the break-even check entirely for history; collapse always when prefix ≥ minCollapsePrefix turns. Trust that amortization wins on average across many sessions. Measure for a week; if average savings go negative, raise the gate.

Pro: simplest code change. Fastest to ship. Generates the most data fastest because every eligible request collapses.
Con: dishonest about per-request economics. On warm-cache-heavy workloads (which is what production Codex traffic actually looks like) this loses money on a non-trivial fraction of requests. "It averages out" is true in expectation but bad UX when the user's specific session is the one that pays the tax.
Status: considered as a measurement-only experiment. Rejected as a default because pixelpipe's reputation is honesty per-request, not per-quarter.

Option D — Cache-bust-driven (event-driven analogue)

Watch incoming requests for the signal that the static-slab cache just flushed (e.g. cache_created_tokens > 0 on a turn where the slab sha didn't change → 4-breakpoint cliff or 5-min idle eviction). On that turn the next call will pay full freight on the entire prefix anyway, so collapse aggressively. On subsequent warm turns, leave the prefix as text.

Pro: maximally honest — only collapses when the host has visible evidence the text path is about to lose. Smallest possible waste.
Con: requires session state (same con as B) plus prior-turn observation. The signal arrives one turn late: by the time we see cache_created, the current turn already paid the cold tax. Best we can do is amortize on the next cold turn, which may be 5 minutes away or never (user closes session).
Status: rejected for v1. The signal arrives too late to drive the current turn's decision. Possibly a useful telemetry signal regardless — knowing when the cache flushed is interesting even if we don't act on it.

Why Option A wins

The decision criteria were, in order:

No session state. Pixelpipe is stateless by design — the same request bytes always produce the same response bytes. State is what turns a library into a service.
No flag. Decisions belong inside the proxy, not on the operator.
Per-request honesty. A request should not pay a tax on the assumption that other requests will recoup it.
Local data only. Don't need to observe the future, don't need to remember the past.

Option A clears all four. B and D fail (1). C fails (3). A's only cost is the speculative render CPU on rejects, which is bounded and measurable.

Why we haven't shipped it yet: needs a count_tokens probe wired through the renderer's output and a host-supplied amortization-horizon constant. Specced; not implemented.

Where to find it in code

Decision: transform.ts:1670 — the historyProfitable predicate
Walking the closed prefix: core/history.ts:findClosedPrefixBoundary
Serialising turns to text: core/history.ts (messagesToText / blocksToText)
Constants: transform.ts:175 (HISTORY_CHARS_PER_TOKEN, HISTORY_DEFAULTS)

Quick start (Node)

npm install
npm run build           # produces dist/node.js
node bin/cli.js         # listens on 127.0.0.1:47821 by default

After editing code, restart in one step:

pnpm run restart                              # graceful SIGTERM → rebuild → start
pnpm run restart -- --no-build                # skip rebuild (dist/ is fresh)
PORT=47822 pnpm run restart                   # override listen port via env

pnpm run restart does, in order:

Lists every running pixelpipe PID (via pgrep) and SIGTERMs them all. Orphans from prior crashed sessions are cleaned up too.
Waits up to 5s for graceful exit (the SIGTERM handler flushes the JSONL tracker). Escalates to SIGKILL only if anything's still alive.
Runs pnpm run build. Build failures abort the restart — the script refuses to start a stale binary. Pass --no-build to skip when you know dist/ is fresh.
Checks the target port is free. If it isn't, names the holding process and refuses to start (cheaper than a crashed Node stacktrace).
execs node bin/cli.js in the foreground so Ctrl-C reaches Node. The proxy takes no behavioral flags — env vars only (see Configuration).

Point Claude Code at it:

ANTHROPIC_BASE_URL=http://127.0.0.1:47821 \
  claude --exclude-dynamic-system-prompt-sections

That's it. Use Claude Code normally.

The --exclude-dynamic-system-prompt-sections flag suppresses the small per-turn variable section so the rendered image stays byte-identical across turns — that's what makes the prompt cache actually hit.

Quick start (Cloudflare Workers)

npx wrangler dev        # local dev on :8787
npx wrangler deploy     # ship to *.workers.dev

Then in Claude Code:

ANTHROPIC_BASE_URL=https://pixelpipe.<your-account>.workers.dev \
  claude --exclude-dynamic-system-prompt-sections

You can attach a custom hostname and route in wrangler.toml.

Configuration

The proxy runs with a single codepath. Every compression mode is on, every break-even threshold is at its measured-best value, and tuning parameters are not user-adjustable. The only configurable surface is where to listen, what to proxy, and where to log — env-var only, no CLI flags.

env var	default	meaning
`PORT`	`47821`	Node only — listen port
`ANTHROPIC_UPSTREAM`	`https://api.anthropic.com`	upstream API base
`PIXELPIPE_LOG`	`~/.pixelpipe/events.jsonl`	persistent event log

In Workers, set the optional upstream API key with:

npx wrangler secret put ANTHROPIC_API_KEY

If unset, the proxy forwards whatever x-api-key the client sent.

Library API

Pixelpipe is also published as a runtime-agnostic transform library so another local proxy can own auth/routing while reusing the Opus 4.7 compression and measurement logic directly:

import {
  transformAnthropicMessages,
  buildCountTokensBodies,
  isPixelpipeSupportedModel,
} from "pixelpipe";

if (isPixelpipeSupportedModel(upstreamModel)) {
  // Run these probes with the host proxy's own auth/transport.
  const baseline = buildCountTokensBodies(originalMessagesBody);

  const result = await transformAnthropicMessages({
    body: originalMessagesBody,
    model: upstreamModel,
  });

  // result.body is the body to forward; result.cache.ownsCacheControl tells
  // the host not to stack a second cache injector on top of pixelpipe.
}

Stable public exports:

pixelpipe — library API plus createProxy for standalone use.
pixelpipe/transform — transformAnthropicMessages(...).
pixelpipe/measurement — count-token probe body builders.
pixelpipe/applicability — Opus 4.7 applicability helpers.
pixelpipe/proxy — standalone Web fetch proxy.

Architecture

src/
├── core/              100% runtime-agnostic (Web Standard APIs only)
│   ├── atlas.ts         (generated) sparse Unicode atlas, base64-inlined
│   ├── png.ts           minimal grayscale PNG encoder
│   ├── render.ts        text → PNG bytes
│   ├── transform.ts     request body rewriter
│   ├── library.ts       public transform wrapper for host proxies
│   ├── measurement.ts   count_tokens probe body builders
│   ├── applicability.ts Opus 4.7 eligibility helpers
│   ├── proxy.ts         the fetch handler
│   └── types.ts         Anthropic API types
├── node.ts            node:http adapter + CLI
└── worker.ts          export default { fetch }

scripts/
├── gen-atlas.ts       build-time: font files → atlas.ts (uses @napi-rs/canvas)
└── build.mjs          esbuild bundler for Node target

assets/
├── Spleen-5x8.otb            primary ASCII/code bitmap font (BSD-2-Clause)
├── SPLEEN_LICENSE.txt        Spleen license
├── Unifont-16.0.04.otf       Unicode fallback (~35k BMP codepoints w/ full-bmp profile)
├── UNIFONT_LICENSE.txt       OFL + GPL-with-font-exception
└── JetBrainsMono-Regular.ttf legacy / ASCII-only fallback (kept on disk)

The atlas is generated at build time from Spleen-5x8.otb (printable ASCII/code) plus Unifont-16.0.04.otf (Unicode fallback), base64-inlined into a .ts file with sparse codepoint + offset tables (binary-packed), and shipped with the bundle. At runtime there are zero external files to read and zero non-Web-Standard imports — that's the only way this works in Workers without per-request asset fetches.

Regenerate the atlas (after swapping fonts, sizes, or codepoint profile):

pnpm run build:atlas                          # default: Spleen 5×8 ASCII + full-bmp Unifont fallback
ATLAS_PROFILE=practical pnpm run build:atlas  # drops Hangul (~24k cp; for Workers free-tier)

Limitations

The bundled hybrid atlas uses Spleen 5×8 for printable ASCII/code and Unifont 8px fallback for ~35k BMP codepoints by default (full-bmp profile): Latin extended, Cyrillic, Greek, CJK Unified Ideographs, Hiragana, Katakana, Hangul, Hebrew, Arabic, math symbols, box-drawing, arrows, Dingbats, Letterlike Symbols, Enclosed Alphanumerics, etc. Drops for codepoints outside the profile (e.g. emoji 😀 — supplementary plane) get counted in events.jsonl#dropped_chars (with the top-20 broken out as dropped_codepoints_top) so you can spot patterns. For Workers free-tier deployments under the 1 MB compressed-bundle cap, switch to ATLAS_PROFILE=practical pnpm run build:atlas (~24k cp; drops Hangul). Right-to-left scripts render left-to-right in source order (no bidi shaping); Devanagari / Thai / similar complex shaping is also unsupported.
The render cell is 5×8 px — the bare Spleen atlas glyph, with DEFAULT_CELL_W_BONUS / DEFAULT_CELL_H_BONUS both 0. 5×8 is the densest cell the eval has cleared on Opus 4.7 given the rest of the render path (packed reflow + grayscale atlas + in-image instruction band) — see Why it's hard → Image density and Eval finding: render the instruction inside the image. With the naïve renderer 5×8 was −16 pp below baseline on dense text and 7×10 was the OCR floor; the three later fixes moved the failure off pixel-pitch and brought 5×8 back to +1.04 pp vs the text-only baseline. The token economics benefit accordingly: a 5×8 character costs ~57 % fewer pixels than the 7×10 cell. The 7×10 cell remains available via eval/-only cellWBonus / cellHBonus overrides for A/B comparison.
Compression sets a 5-minute prompt-cache TTL. Adding cache_control: ephemeral causes warm-cache rotation, not eviction.
A 5KB break-even point: if input is < MIN_COMPRESS_CHARS chars we skip compression entirely (overhead would exceed savings).
Per-machine font: regenerate the atlas if you swap fonts. The generated src/core/atlas.ts is checked in so consumers don't need @napi-rs/canvas to install.
Workers CPU limit: this is fine for free-tier (10ms CPU) on small prompts; large prompts (>30K chars) may need the paid tier.

Development

npm install
npm run dev:node              # tsx watch on src/node.ts
npm run dev:worker            # wrangler dev
npm run test                  # vitest
npm run test:watch
npm run typecheck             # tsc --noEmit
pnpm run build:atlas          # regenerate src/core/atlas.ts from OTF
npm run build                 # build dist/node.js
npm run deploy:worker         # wrangler deploy

License

MIT.

README.md Unescape Escape

pixelpipe