pixelpipe

Turn Claude's tool-result text into compact PNGs before it ever reaches the model. Anthropic charges per token; vision tokens for a dense 1568×1568 image are dramatically cheaper than the same content delivered as transcript text. pixelpipe is the encoder that exploits that gap.

It is a small, focused TypeScript library — no daemon, no MCP wiring, no opinions about transport. You hand it a string, it hands you one or more ready-to-send PNG buffers.

---

Status

Experimental, and the cost math is workload-dependent — read this before relying on it.

What it does. Rewrites Claude Code tool-result / history text into dense PNGs. On a live, multi-session run against real Claude Code traffic it measured ~68% fewer input tokens (856k → 277k over the session), because that traffic is token-dense (~1 char/token: JSON, code, tool output, hashes) and a dense image packs ~3.1 chars per image-token. On sparse English prose (~3.5 chars/token) the same images lose money — so the savings depend entirely on what you feed it.

What it is. A lossy, recency-graded gist compressor. Recent turns stay text; older bulk history becomes images. A needle-in-haystack eval recovered 0/15 exact 12-char hex strings from rendered images across two model generations — so imaged content is safe to skim by gist but cannot be relied on for verbatim recall, and the failure mode is silent confabulation (it returns a plausible wrong value, not an error). Do not image anything you may need back byte-exact (IDs, hashes, secrets, exact numbers) until a verbatim-risk guard keeps those blocks as text.

Model scope. Opus 4.7 and newer (4.x) only, enforced in both the library (isPixelpipeSupportedModel) and the proxy. Older Opus (≤ 4.6) and non-Opus families are not enabled.

---

Benchmarks (reproducible)

Does imaging preserve answers? We measured it three ways on claude-opus-4-8, and — unlike a marketing table — we report where it breaks.

1. Reading fidelity, clean — novel random-number problems (zero memorization possible):

test	N	baseline (text)	pixelpipe (image)	delta
novel arithmetic, random numbers	100	100%	93%	−7pp

This is the honest reading number. With the answer un-memorizable, the model reads pixelpipe's render correctly ~93% of the time on short content; the other ~7% are misreads (10200→9400, 7873→7793) or come back unreadable.

2. GSM8K — the standard suite, but it's in training data:

benchmark	N	baseline (text)	pixelpipe (image)	tokens/problem
GSM8K (math)	100	97%	96%	61 → 38 (−38%)

GSM8K looks near-lossless (−1pp), but that's inflated — the model recognizes memorized problems and recalls the answer even when it misreads the image. The novel test strips that out: the real reading cost is ~7pp, not ~1.

3. Where it fails completely — dense / verbatim:

test	baseline (text)	pixelpipe (image)
verbatim recall — exact 12-char hex from a dense render	15/15 (100%)	0/15 (0%)

So pixelpipe reads short, readable content at ~93% (a real ~7% misread tax) and silently fails on dense walls and exact recall. The number you get depends entirely on what you feed it.

Reproduce: eval/gsm8k/ (GSM8K + the novel reading test) and eval/needle-haystack/ (the verbatim failure). Full analysis: FINDINGS.md.

---

How it works

tool_result string  ──►  wrapLines  ──►  renderTextToPngs  ──►  PNG[]

1. Wrap the input at a column width that fits 1568 px wide.
2. Pack as many lines as fit into a single readable image (≈ DENSE_CONTENT_CHARS_PER_IMAGE = 5000 chars per page).
3. Render each page to a PNG via node-canvas.
4. Return the array. Callers attach the PNGs to the user message and drop the original text.

The math

A Claude 1568×1568 image costs ≈ 1568 vision tokens (Anthropic, 2026-04-16). At ≈ 6 readable characters per square monospace glyph, that page holds ≈ 5 000 text chars. Same content as plain text: ≈ 1 250 text tokens. So plain text is cheaper unless the model treats vision tokens as much fatter than text tokens — which Opus 4.6/4.7 effectively do on cold-miss cached transcripts.

We measure rather than guess. The runtime estimator (estimateImageCount) tells the caller how many images a string would produce; the caller's gate decides whether that beats sending text. Built-in defaults are model-aware: Opus 4.7 uses 2.0 chars/token for slab/history gates, while Opus 4.6 uses the older, more conservative 2.5 chars/token default unless the host supplies an empirical override.

Why we don't just render one giant image

Earlier versions packed everything into a single 1568×1568 PNG. With long inputs this either (a) shrank the font below OCR-legibility or (b) used multi-column packing that broke OCR ordering on the encoder side.

The current behaviour:

input size	output
≤ minToolResultChars (~6 000)	not rendered — caller sends as text
moderate (≤ 5 000/page)	one 1568×~480 PNG
long	N pages, each 1568×~480, paginated

Every page renders at the same font size and column width. Page heights scale with content; no more dense walls of unreadable text.

Single-column vs. multi-column

Multi-column packing (two columns side-by-side on one page) is supported but disabled by default. Reason: the OCR / vision encoder reads in row order, so two columns silently corrupt sequence integrity. The code is preserved behind numCols > 1; do not enable it unless you have measured both faithfulness and savings.

---

Quick start (Node)

import { renderTextToPngs } from "pixelpipe";

const pngs = await renderTextToPngs(toolResultText);
// pngs: Buffer[]  — attach to the next user turn

Quick start (Cloudflare Workers)

renderTextToPngs works in Workers via the WASM build of node-canvas shipped under dist/wasm/. Set nodejs_compat in wrangler.toml.

import { renderTextToPngs } from "pixelpipe";

export default {
  async fetch(req: Request) {
    const text = await req.text();
    const pngs = await renderTextToPngs(text);
    return new Response(pngs[0], { headers: { "content-type": "image/png" } });
  },
};

---

Library API

// Top-level: render a string to one or more PNG pages.
renderTextToPngs(text: string, cols?: number, style?: RenderStyle): Promise<Buffer[]>

// Lower-level helpers (exported for callers that want to gate themselves):
estimateImageCount(text: string, cols?: number): number
shrinkColsToContent(text: string, cols: number): number
wrapLines(text: string, cols: number, markerScale?: number): string[]

Constants

name	value	meaning
READABLE_CHARS_PER_IMAGE	6 000	upper bound on chars packed into one page
MIN_TOO_L_RESULT_CHARS	6 000	inputs below this should not be rendered
MIN_REMINDER_CHARS	6 000	gate for adding "(see image)" reminder text
DEFAULT_COLS	100	column width when caller doesn't override
MAX_HEIGHT_PX	1 568	page height ceiling
MAX_WIDTH_PX	1 568	page width

---

Configuration

There is none in the library itself. Callers (e.g. ocproxy) decide:

• whether to render this particular tool_result at all
• what cols to pass (often DEFAULT_COLS is fine)
• what to do with the PNGs (attach, cache, etc.)

---

Architecture

src/core/
  render.ts        renderTextToPngs, wrapLines, encodeGrayPng
  transform.ts    estimateImageCount, transformAnthropicMessages,
                  textToImageBlocks, shrinkColsToContent
  library.ts       public re-exports → dist/core/index.js

src/server/ and src/dashboard* are not part of the library; they are tools used during development and for the demo dashboard.

---

Development

pnpm install
pnpm run typecheck      # 315 tests pass
pnpm test
pnpm run build           # regenerates dist/

Tests of interest:

• tests/paging.test.ts — page-count contract across sizes
• tests/render.test.ts — wrap / shrink / gate behaviour

The paging contract: with the 6 000-char readable cap, geometry is ~480 px tall per page, and estimateImageCount returns ceil(chars / 6 000) once the input clears the profitability gate.

---

Limitations

• Only ASCII / Latin-1 has been seriously tested. Wide CJK glyphs work but their markerScale heuristics are conservative.
• node-canvas is a native dep on Node and a WASM dep on Workers. The Workers build is larger.
• No streaming. Rendering is per-tool_result.
• Profitability is workload-specific, not just model-specific. It wins on token-dense content (code, JSON, tool output, hashes ~1 char/token) and loses on sparse prose (~3.5 chars/token). Enabled for Opus 4.7+ callers.
• Verbatim recall is unreliable. Exact strings inside imaged content (0/15 in eval) can be silently confabulated — a plausible wrong value, not an error. Keep anything you need byte-exact as text; pixelpipe is a lossy gist tier, not a lossless store. A verbatim-risk guard (skip blocks with unique IDs / hashes / exact values) is not yet built.

---

License

MIT.