From c35b3ab2d489fe36ca9db5f3e1379421c885a136 Mon Sep 17 00:00:00 2001 From: teamchong <25894545+teamchong@users.noreply.github.com> Date: Tue, 9 Jun 2026 19:10:49 -0400 Subject: [PATCH] =?UTF-8?q?docs:=20rewrite=20README=20for=20readability=20?= =?UTF-8?q?=E2=80=94=20hook=20first,=20real=20render=20example,=20honest-c?= =?UTF-8?q?aveats=20section?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 300 +++++++++++++++--------------------------------------- 1 file changed, 81 insertions(+), 219 deletions(-) diff --git a/README.md b/README.md index e176964..844ab6a 100644 --- a/README.md +++ b/README.md @@ -1,266 +1,128 @@ # pxpipe -Turn Claude or GPT static context into compact PNGs before it ever reaches the -model. Text tokens are expensive; vision tokens for a dense 1568×1568 image can -be dramatically cheaper than the same content delivered as transcript text. -pxpipe is the encoder that exploits that gap. +**Cut Claude Code input-token spend by rendering old context as images.** -It is a small, focused TypeScript library — no daemon, no MCP wiring, no -opinions about transport. You hand it a string, it hands you one or more -ready-to-send PNG buffers. +Anthropic bills a 1568px-wide image at a flat rate regardless of how much text +is inside it. Dense content (code, JSON, tool output) packs ~3.1 chars per +image-token vs ~1 char per text-token on real Claude Code traffic. pxpipe is a +local proxy that exploits that gap: it rewrites the bulky middle of your +conversation into compact PNGs before the request leaves your machine. ---- +Running against real Claude Code sessions, the production log shows +**77% input tokens saved across 6,691 requests** (3.21B baseline → 735M +actual). Single sessions measure ~68%. -## Status +This is what the model sees instead of text: -Experimental, and the cost math is **workload-dependent** — read this before -relying on it. +![example: 5,000 chars of TypeScript rendered as one 908×880 PNG](docs/assets/example-render.png) -**What it does.** Rewrites Claude Code tool-result / history text into dense -PNGs. On a live, multi-session run against real Claude Code traffic it -measured **~68% fewer input tokens** (856k → 277k over the session), and the -cumulative production log now shows **77% saved across 6,691 compressed -requests** (3.21B baseline tokens → 735M actual, as of 2026-06-09), because -that traffic is token-dense (~1 char/token: JSON, code, tool output, hashes) -and a dense image packs ~3.1 chars per image-token. On sparse English prose -(~3.5 chars/token) the same images *lose* money — so the savings depend -entirely on what you feed it. +*5,000 characters of source code — ≈1,250 text tokens as a transcript, ≈530 +image tokens as this PNG. The model reads it at 100/100 on a clean eval (see +benchmarks).* -**What it is.** A lossy, recency-graded **gist** compressor. Recent turns stay -text; older bulk history becomes images. A needle-in-haystack eval recovered -**0/15** exact 12-char hex strings from rendered images across two model -generations — so imaged content is safe to skim by gist but **cannot be -relied on for verbatim recall**, and the failure mode is *silent -confabulation* (it returns a plausible wrong value, not an error). Do not -image anything you may need back byte-exact (IDs, hashes, secrets, exact -numbers) until a verbatim-risk guard keeps those blocks as text. +## Try it (30 seconds) -**Model scope.** Fable 5 (`claude-fable-5`) only on the Anthropic route, -enforced in both the library (`isPxpipeSupportedModel`) and the proxy. -Opus (4.7/4.8, the original measured scope) was disabled 2026-06-09: Fable 5 -reads renders at 100/100 on the novel-arithmetic eval vs Opus 4.8's 93/100, -with identical image billing (same Opus 4.7-line tokenizer, verified by direct -measurement) — so the ~7% Opus read tax is no longer worth carrying. An OpenAI -`/v1/chat/completions` route exists for GPT 5.5 (`gpt-5.5*`) but is not the -focus and is unmeasured beyond smoke tests. Mythos 5 is unmeasured (no access). +```bash +npx pxpipe # proxy on 127.0.0.1:47821 +ANTHROPIC_BASE_URL=http://localhost:47821 claude # point Claude Code at it +``` ---- +Open for a live dashboard: tokens saved, per-session +stats, every text→image conversion side by side, and a kill switch. + +Nothing else changes. Responses stream normally — pxpipe only compresses the +*request* (your context going up), never the model's output. Recent turns stay +text; only older bulk history is imaged. + +## The honest part — read before relying on it + +**It is lossy.** pxpipe is a *gist* tier, not a lossless store. In a +needle-in-haystack eval, exact 12-char hex strings inside dense imaged content +came back **0/15** on Opus and 3/4 on Fable 5 — and the failure mode is +*silent confabulation*: a plausible wrong value, not an error. Anything you +need back byte-exact (IDs, hashes, secrets, exact numbers) must stay text. +Recent turns do; a dedicated verbatim-risk guard is not built yet. + +**Savings are workload-dependent.** It wins on token-dense content +(~1 char/token: code, JSON, hashes) and *loses money* on sparse English prose +(~3.5 chars/token). The built-in gate only images content where the math wins, +calibrated against N=391 production rows. + +**Model scope: Fable 5 only** (`claude-fable-5`), enforced in library and +proxy. Opus 4.7/4.8 was the original scope but misread ~7% of renders +(`10200`→`9400`), so it was disabled once Fable 5 hit 100/100 with identical +image billing. Everything else passes through untouched. ## Benchmarks (reproducible) -**One number: on short, readable content Fable 5 reads pxpipe's render -100% of the time, at ~38% fewer tokens.** Measured clean — with novel -random-number problems it cannot have memorized: +Measured with novel random-number problems the model cannot have memorized: | test | N | text | pxpipe (image) | tokens | |---|---:|---:|---:|---| -| novel arithmetic, `claude-fable-5` (2026-06-09) | 100 | 100% | **100%** | **−38%** | +| novel arithmetic, `claude-fable-5` | 100 | 100% | **100%** | **−38%** | | novel arithmetic, `claude-opus-4-8` | 100 | 100% | 93% | −38% | +| verbatim 12-char hex recall, dense render, Opus | 15 | 15/15 | **0/15** | — | +| verbatim 12-char hex recall, dense render, Fable 5 | 4 | — | 3/4 | — | -The Opus ~7% gap was real misreads (`10200`→`9400`, `7873`→`7793`) — that read -tax is why Opus is now disabled and the gate is Fable-only. - -**The boundary** — push to dense, exact-recall content and it degrades: - -| test | text | pxpipe (image) | -|---|---:|---:| -| verbatim recall — 12-char hex from a *dense* render, Opus | 15/15 | **0/15** | -| verbatim recall — 12-char hex, dense JSON render, Fable 5 (n=4, smaller page) | — | **3/4** | - -Fable 5 dramatically improves verbatim recall but still produces single-glyph -silent misreads (`125f9e6e1c77`→`125f9e6a1c77`, `cc33ae67`→`cc33a867`), so the -rule stands: do not image anything you need back byte-exact. Full analysis in -[`FINDINGS.md`](FINDINGS.md). - -We also ran the standard **GSM8K** suite: 96% imaged. But GSM8K is in training -data, so the model recalls memorized answers through its own misreads — inflating -the score ~3pp over the clean novel number above, which is why we don't lead with -it. Reproduce: [`eval/gsm8k/`](eval/gsm8k/) · [`eval/needle-haystack/`](eval/needle-haystack/). - ---- +We also ran GSM8K: 96% imaged. But GSM8K is in training data — the model +recalls memorized answers through its own misreads, inflating the score — so we +lead with the clean novel-number eval instead. Reproduce: +[`eval/gsm8k/`](eval/gsm8k/) · [`eval/needle-haystack/`](eval/needle-haystack/) · +full analysis in [`FINDINGS.md`](FINDINGS.md). ## How it works ``` -tool_result string ──► wrapLines ──► renderTextToPngs ──► PNG[] +tool_result string ──► wrap at 1568px-wide columns ──► pack ~5,000 chars/page ──► PNG[] ``` -1. **Wrap** the input at a column width that fits 1568 px wide. -2. **Pack** as many lines as fit into a single readable image - (≈ `DENSE_CONTENT_CHARS_PER_IMAGE = 5000` chars per page). -3. **Render** each page to a PNG via `node-canvas`. -4. **Return** the array. Callers attach the PNGs to the user message and - drop the original text. +The proxy intercepts `/v1/messages`, rewrites eligible bulk history into image +blocks, splices them back cache-friendly (static prefix preserved, so prompt +caching keeps working), and forwards. Per-request events log to +`~/.pxpipe/events.jsonl`. -### The math +The economics: a 1568×1568 image costs ≈1,568 vision tokens and holds ≈5,000 +readable chars (≈1,250 text tokens) — so plain text is cheaper *unless* your +text is token-dense. Claude Code transcripts are (observed 1.91 chars/token, +N=391). The runtime estimator (`estimateImageCount`) plus a chars/token gate +decides per-request; sparse prose is left as text. -A Claude 1568×1568 image costs ≈ 1568 vision tokens (Anthropic, 2026-04-16). -At ≈ 6 readable characters per square monospace glyph, that page holds -≈ 5 000 text chars. Same content as plain text: ≈ 1 250 text tokens. So -plain text is cheaper *unless* the model treats vision tokens as much -fatter than text tokens — which they effectively do on cold-miss -cached transcripts. - -We measure rather than guess. The runtime estimator -(`estimateImageCount`) tells the caller how many images a string would -produce; the caller's gate decides whether that beats sending text. The -built-in gate constant is `2.0` chars/token, calibrated against N=391 -production rows (observed 1.91), unless the host supplies an empirical -override via `opts.charsPerToken`. - -### Why we don't just render one giant image - -Earlier versions packed everything into a single 1568×1568 PNG. With long -inputs this either (a) shrank the font below OCR-legibility or (b) used -multi-column packing that broke OCR ordering on the encoder side. - -The current behaviour: - -| input size | output | -|---------------------------|------------------------------------------| -| ≤ `minToolResultChars` (~6 000) | not rendered — caller sends as text | -| moderate (≤ 5 000/page) | one 1568×~480 PNG | -| long | N pages, each 1568×~480, paginated | - -Every page renders at the same font size and column width. Page heights -scale with content; no more dense walls of unreadable text. - -### Single-column vs. multi-column - -Multi-column packing (two columns side-by-side on one page) is supported -but disabled by default. Reason: the OCR / vision encoder reads in row -order, so two columns silently corrupt sequence integrity. The code is -preserved behind `numCols > 1`; do not enable it unless you have measured -both faithfulness *and* savings. - ---- - -## Quick start (CLI proxy — Claude Code) - -The fastest way to try it: run the local proxy and point Claude Code at it. -It transparently compresses eligible `/v1/messages` bodies and passes -everything else through untouched. - -```bash -npx pxpipe # listens on http://127.0.0.1:47821 -ANTHROPIC_BASE_URL=http://localhost:47821 claude -``` - -A live dashboard (tokens saved, per-session stats, compression kill switch) -is served at the same address: . Per-request events -are logged to `~/.pxpipe/events.jsonl`. - -## Quick start (Node) +## Library use (no proxy) ```ts -import { renderTextToPngs } from "pxpipe"; +import { renderTextToPngs, estimateImageCount } from "pxpipe"; -const pngs = await renderTextToPngs(toolResultText); -// pngs: Buffer[] — attach to the next user turn +const pngs = await renderTextToPngs(toolResultText); // Buffer[] — attach to the next user turn ``` -## Proxy environment variables - -An OpenAI-compatible route (`/v1/chat/completions`, GPT 5.5 only) also exists -on the same port — we tried it, it works in smoke tests, but it is unmeasured -and not the focus; Fable 5 via the Anthropic route is. - - - -| name | default | meaning | -|---|---|---| -| `ANTHROPIC_UPSTREAM` | `https://api.anthropic.com` | Upstream for `/v1/messages` | -| `OPENAI_UPSTREAM` | `https://api.openai.com` | Upstream for `/v1/chat/completions` | -| `OPENAI_API_KEY` | unset | Optional OpenAI key override; otherwise client `Authorization` is forwarded | - ---- - -## Library API - ```ts -// Top-level: render a string to one or more PNG pages. renderTextToPngs(text: string, cols?: number, style?: RenderStyle): Promise - -// Lower-level helpers (exported for callers that want to gate themselves): -estimateImageCount(text: string, cols?: number): number -shrinkColsToContent(text: string, cols: number): number +estimateImageCount(text: string, cols?: number): number // gate yourself wrapLines(text: string, cols: number, markerScale?: number): string[] ``` -### Constants - -| name | value | meaning | -|---------------------------------|--------|------------------------------------------------| -| `READABLE_CHARS_PER_IMAGE` | 50 000 | hard ceiling on chars packed into one page | -| `DENSE_CONTENT_CHARS_PER_IMAGE` | 5 000 | target chars/page for dense tool-result pages | -| `DENSE_CONTENT_COLS` | 180 | column width for dense pages | -| `DEFAULT_COLS` | 313 | column width when caller doesn't override | -| `MAX_HEIGHT_PX` | 1 568 | page height ceiling | - ---- - -## Configuration - -There is none in the library itself. Callers (e.g. the bundled proxy) decide: - -* whether to render this particular tool_result at all -* what `cols` to pass (often `DEFAULT_COLS` is fine) -* what to do with the PNGs (attach, cache, etc.) - ---- - -## Architecture - -``` -src/core/ - render.ts renderTextToPngs, wrapLines, encodeGrayPng - transform.ts estimateImageCount, transformAnthropicMessages, - textToImageBlocks, shrinkColsToContent - library.ts public re-exports → dist/core/index.js -``` - -`src/server/` and `src/dashboard*` are *not* part of the library; they -are tools used during development and for the demo dashboard. - ---- +| constant | value | meaning | +|---|---|---| +| `DENSE_CONTENT_CHARS_PER_IMAGE` | 5 000 | target chars per page | +| `READABLE_CHARS_PER_IMAGE` | 50 000 | hard ceiling per page | +| `DEFAULT_COLS` | 313 | column width | +| `MAX_HEIGHT_PX` | 1 568 | page height ceiling | ## Development ```bash -pnpm install -pnpm run typecheck -pnpm test # 323 tests -pnpm run build # regenerates dist/ +pnpm install && pnpm test # 323 tests +pnpm run build # regenerates dist/ ``` -Tests of interest: - -* `tests/paging.test.ts` — page-count contract across sizes -* `tests/render.test.ts` — wrap / shrink / gate behaviour - -The paging contract: with the 6 000-char readable cap, geometry is -~480 px tall per page, and `estimateImageCount` returns ceil(chars / 6 000) -once the input clears the profitability gate. - ---- - ## Limitations -* Only ASCII / Latin-1 has been seriously tested. Wide CJK glyphs work - but their `markerScale` heuristics are conservative. -* `node-canvas` is a native dep on Node and a WASM dep on Workers. The - Workers build is larger. -* No streaming. Rendering is per-tool_result. -* Profitability is **workload-specific**, not just model-specific. It wins on - token-dense content (code, JSON, tool output, hashes ~1 char/token) and - loses on sparse prose (~3.5 chars/token). Enabled for Fable 5 callers. -* **Verbatim recall is unreliable.** Exact strings inside imaged content (0/15 - in eval) can be silently confabulated — a plausible wrong value, not an - error. Keep anything you need byte-exact as text; pxpipe is a lossy gist - tier, not a lossless store. A verbatim-risk guard (skip blocks with unique - IDs / hashes / exact values) is not yet built. - ---- +* **Lossy** — see "the honest part" above. Verbatim recall from images is unreliable. +* Render latency: encoding PNGs adds time to large requests before they leave + (partly offset by the model ingesting fewer tokens). Responses stream normally. +* ASCII/Latin-1 well tested; CJK works but conservatively. +* `node-canvas` native dep on Node. +* Fable 5 only. An unmeasured GPT 5.5 route exists (`OPENAI_BASE_URL=http://127.0.0.1:47821/v1`) — smoke-tested, not the focus. ## License