pxpipe/README.md

pixelpipe

Turn Claude's tool-result text into compact PNGs before it ever reaches the model. Anthropic charges per token; vision tokens for a dense 1568×1568 image are dramatically cheaper than the same content delivered as transcript text. pixelpipe is the encoder that exploits that gap.

It is a small, focused TypeScript library — no daemon, no MCP wiring, no opinions about transport. You hand it a string, it hands you one or more ready-to-send PNG buffers.

---

Status

Experimental, and the cost math is workload-dependent — read this before relying on it.

What it does. Rewrites Claude Code tool-result / history text into dense PNGs. On a live, multi-session run against real Claude Code traffic it measured ~68% fewer input tokens (856k → 277k over the session), because that traffic is token-dense (~1 char/token: JSON, code, tool output, hashes) and a dense image packs ~3.1 chars per image-token. On sparse English prose (~3.5 chars/token) the same images lose money — so the savings depend entirely on what you feed it.

What it is. A lossy, recency-graded gist compressor. Recent turns stay text; older bulk history becomes images. A needle-in-haystack eval recovered 0/15 exact 12-char hex strings from rendered images across two model generations — so imaged content is safe to skim by gist but cannot be relied on for verbatim recall, and the failure mode is silent confabulation (it returns a plausible wrong value, not an error). Do not image anything you may need back byte-exact (IDs, hashes, secrets, exact numbers) until a verbatim-risk guard keeps those blocks as text.

Model scope. Opus 4.7 and newer (4.x) only, enforced in both the library (isPixelpipeSupportedModel) and the proxy. Older Opus (≤ 4.6) and non-Opus families are not enabled.

---

Benchmarks (reproducible)

One number: on short, readable content the model reads pixelpipe's render ~93% of the time, at ~38% fewer tokens. Measured clean — with novel random-number problems it cannot have memorized, on claude-opus-4-8:

test	N	text	pixelpipe (image)	tokens
novel arithmetic (un-memorizable)	100	100%	93%	−38%

The ~7% gap is real misreads (10200→9400, 7873→7793), not noise — imaging is lossy even where it mostly works.

The boundary — push to dense, exact-recall content and it collapses:

test	text	pixelpipe (image)
verbatim recall — 12-char hex from a dense render	15/15	0/15

~93% on short / readable, 0% on dense / exact-recall. There is no free lunch — full analysis in FINDINGS.md.

_{We also ran the standard GSM8K suite: 96% imaged. But GSM8K is in training data, so the model recalls memorized answers through its own misreads — inflating the score ~3pp over the clean novel number above, which is why we don't lead with it. Reproduce: eval/gsm8k/ · eval/needle-haystack/.}

---

How it works

tool_result string  ──►  wrapLines  ──►  renderTextToPngs  ──►  PNG[]

1. Wrap the input at a column width that fits 1568 px wide.
2. Pack as many lines as fit into a single readable image (≈ DENSE_CONTENT_CHARS_PER_IMAGE = 5000 chars per page).
3. Render each page to a PNG via node-canvas.
4. Return the array. Callers attach the PNGs to the user message and drop the original text.

The math

A Claude 1568×1568 image costs ≈ 1568 vision tokens (Anthropic, 2026-04-16). At ≈ 6 readable characters per square monospace glyph, that page holds ≈ 5 000 text chars. Same content as plain text: ≈ 1 250 text tokens. So plain text is cheaper unless the model treats vision tokens as much fatter than text tokens — which Opus 4.6/4.7 effectively do on cold-miss cached transcripts.

We measure rather than guess. The runtime estimator (estimateImageCount) tells the caller how many images a string would produce; the caller's gate decides whether that beats sending text. Built-in defaults are model-aware: Opus 4.7 uses 2.0 chars/token for slab/history gates, while Opus 4.6 uses the older, more conservative 2.5 chars/token default unless the host supplies an empirical override.

Why we don't just render one giant image

Earlier versions packed everything into a single 1568×1568 PNG. With long inputs this either (a) shrank the font below OCR-legibility or (b) used multi-column packing that broke OCR ordering on the encoder side.

The current behaviour:

input size	output
≤ minToolResultChars (~6 000)	not rendered — caller sends as text
moderate (≤ 5 000/page)	one 1568×~480 PNG
long	N pages, each 1568×~480, paginated

Every page renders at the same font size and column width. Page heights scale with content; no more dense walls of unreadable text.

Single-column vs. multi-column

Multi-column packing (two columns side-by-side on one page) is supported but disabled by default. Reason: the OCR / vision encoder reads in row order, so two columns silently corrupt sequence integrity. The code is preserved behind numCols > 1; do not enable it unless you have measured both faithfulness and savings.

---

Quick start (Node)

import { renderTextToPngs } from "pixelpipe";

const pngs = await renderTextToPngs(toolResultText);
// pngs: Buffer[]  — attach to the next user turn

Quick start (Cloudflare Workers)

renderTextToPngs works in Workers via the WASM build of node-canvas shipped under dist/wasm/. Set nodejs_compat in wrangler.toml.

import { renderTextToPngs } from "pixelpipe";

export default {
  async fetch(req: Request) {
    const text = await req.text();
    const pngs = await renderTextToPngs(text);
    return new Response(pngs[0], { headers: { "content-type": "image/png" } });
  },
};

---

Library API

// Top-level: render a string to one or more PNG pages.
renderTextToPngs(text: string, cols?: number, style?: RenderStyle): Promise<Buffer[]>

// Lower-level helpers (exported for callers that want to gate themselves):
estimateImageCount(text: string, cols?: number): number
shrinkColsToContent(text: string, cols: number): number
wrapLines(text: string, cols: number, markerScale?: number): string[]

Constants

name	value	meaning
READABLE_CHARS_PER_IMAGE	6 000	upper bound on chars packed into one page
MIN_TOO_L_RESULT_CHARS	6 000	inputs below this should not be rendered
MIN_REMINDER_CHARS	6 000	gate for adding "(see image)" reminder text
DEFAULT_COLS	100	column width when caller doesn't override
MAX_HEIGHT_PX	1 568	page height ceiling
MAX_WIDTH_PX	1 568	page width

---

Configuration

There is none in the library itself. Callers (e.g. ocproxy) decide:

• whether to render this particular tool_result at all
• what cols to pass (often DEFAULT_COLS is fine)
• what to do with the PNGs (attach, cache, etc.)

---

Architecture

src/core/
  render.ts        renderTextToPngs, wrapLines, encodeGrayPng
  transform.ts    estimateImageCount, transformAnthropicMessages,
                  textToImageBlocks, shrinkColsToContent
  library.ts       public re-exports → dist/core/index.js

src/server/ and src/dashboard* are not part of the library; they are tools used during development and for the demo dashboard.

---

Development

pnpm install
pnpm run typecheck      # 315 tests pass
pnpm test
pnpm run build           # regenerates dist/

Tests of interest:

• tests/paging.test.ts — page-count contract across sizes
• tests/render.test.ts — wrap / shrink / gate behaviour

The paging contract: with the 6 000-char readable cap, geometry is ~480 px tall per page, and estimateImageCount returns ceil(chars / 6 000) once the input clears the profitability gate.

---

Limitations

• Only ASCII / Latin-1 has been seriously tested. Wide CJK glyphs work but their markerScale heuristics are conservative.
• node-canvas is a native dep on Node and a WASM dep on Workers. The Workers build is larger.
• No streaming. Rendering is per-tool_result.
• Profitability is workload-specific, not just model-specific. It wins on token-dense content (code, JSON, tool output, hashes ~1 char/token) and loses on sparse prose (~3.5 chars/token). Enabled for Opus 4.7+ callers.
• Verbatim recall is unreliable. Exact strings inside imaged content (0/15 in eval) can be silently confabulated — a plausible wrong value, not an error. Keep anything you need byte-exact as text; pixelpipe is a lossy gist tier, not a lossless store. A verbatim-risk guard (skip blocks with unique IDs / hashes / exact values) is not yet built.

---

License

MIT.