Add native Google AI Studio transformation, measured token accounting, dashboard support, and a production 312×728 profile for Gemini 3.6 Flash.
Gemini uses 26% fewer vision tokens than Fable 5 per full page and matches or exceeds Fable across the production-profile quality suite.
* fix(render): glyph surgery so Spleen 5x8 K no longer reads as H
The stock Spleen K was H with one crossbar pixel removed (Hamming 1), the
worst confusable pair in the atlas. Because the API downscales images and the
model infers rather than scans, that difference vanishes and K is silently
misread as H - with no signal of uncertainty.
gen-atlas.ts now repaints K with a diagonal-legged bitmap: Hamming 8 from H,
and >=6 from every other ASCII glyph. Scoped to the narrow Spleen 5x8 primary
cell, so the JetBrains-Mono and CJK-fallback atlases are untouched. Zero
token-cost change - the glyph occupies the same 5x8 box.
Guarded by a regression test asserting no alphanumeric pair sits below
Hamming 2. Reproduce with: npx tsx eval/glyph-matrix/demo-glyph.mts
Closes the "optional glyph surgery" item in the 2026-07-01 legibility audit.
scripts/build.mjs spawned `pnpm exec tsc` with `spawnSync('pnpm', ...,
{ shell: false })`. On Windows the launcher is `pnpm.cmd`, which spawnSync
cannot run without a shell: it returns `status: null` (ENOENT), so the
`process.exit(tsc.status ?? 1)` line exited the build with code 1 and no
output — `pnpm run build` failed silently.
Run tsc's JS entry with the current Node binary instead
(`node <resolved typescript/bin/tsc>`), mirroring the existing --version
smoke check. This avoids the .cmd spawn problem (and the shell:true DEP0190
arg-escaping warning) entirely, is cross-platform, and removes the build's
assumption that pnpm is the caller. Also surface spawn errors explicitly
instead of collapsing them into a bare exit 1.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Ship Grok 5×8 white/IDS packing and universal IDS/factsheet plumbing while
keeping DEFAULT_MODEL_BASES Fable-only: live pure-image Grok is not
Fable-level; image+factsheet is the working exact path.
Chart shows Grok as a text-window series only (no Grok+pxpipe overlay) and
Fable as the measured pxpipe density point. Add Codex-path quality harnesses
and receipts (multi-seed, fix-matrix, shipped smoke).
Plot characters-held per frontier context window, 2018-2026, ending at
the same Fable 5 1M window read through pxpipe images: a measured ~18M
chars vs the ~4M ceiling every text line converges on.
The chart is generated, not drawn: gen-context-chart.ts measures
chars/vision-token from a live pipeline render at generation time and
derives the 4.6x multiplier from that, so the claim regenerates with
the code it describes. Historical points carry their sourcing as code
comments — including the two a reviewer would check: Gemini 1.5 Pro is
plotted at its June 2024 2M peak (the line honestly dips to 2.5 Pro's
1M), and Sonnet 4 [1m] appears as Anthropic's first 1M window so the
Claude line doesn't imply Fable 5 was first.
Several families share /v1/responses. Cost, geometry, cache rates, and
whether imaging is even allowed must follow the model that serves the
request. Otherwise Claude and Grok inherit GPT defaults, the gate lies,
and the dashboard reports nonsense.
Claude on Responses:
- Bill images by Anthropic pixel area; cache 0.1x, output 5x.
- Use Anthropic page geometry (312 cols x 728 px). GPT height overstated
image cost ~2.6x and flipped every Opus slab to not_profitable, so
enabled Claude stayed text-only with blank As text / Saved.
History profitability gate:
- Reflow packs hard newlines with an inline ↵ glyph. countVisualRows
still treated ↵ as a row break, so estimateImageCount overstated pages
~6x on reflowed history and collapsed Claude/Grok history never fired.
Match the renderer: only hard newlines start a visual row.
Grok:
- Keep production 5x8 packing and a verbatim fact-sheet for OCR-hard
tokens (paths/hex/ports/camelCase). Measured ~1000 image tok/MPix;
cache 0.25x, output 3x.
- Leave Grok out of DEFAULT_MODEL_BASES (opt-in only, same bar as Opus).
Pure-image exact OCR fails at 5x8; do not image it silently.
Also thread optional per-model render style through the Responses path
and allow eval-only atlas overrides in gen-atlas.
printVersion() logged `process.env.npm_package_version ?? '0.2.0'`. npm only
populates npm_package_version inside its own run-script env, so an end user
running `npx pxpipe-proxy --version` or a globally-installed `pxpipe` got the
stale hardcoded fallback (0.2.0) instead of the real version — the code comment
even claimed an esbuild.define that was never wired up.
- scripts/build.mjs reads package.json and inlines the version via esbuild
`define: { __PXPIPE_VERSION__ }`, then smoke-checks the built binary
(`node dist/node.js --version` must equal package.json) and fails the build
on mismatch, so a broken injection can never ship.
- src/node.ts reads the injected constant behind a `typeof` guard (safe under
tsx, where it is undefined), falling back to npm_package_version then
'unknown' — never a stale release number.
Verified: node dist/node.js --version now prints 0.8.0.
Fable 5 / Opus 4.8 accept up to 2576px / 4784 visual tokens per image, but a
request with >20 images (pxpipe always sends many) is held to the stricter
<=2000px/side rule, so the real ceiling is ~1932x1932 (1928x1928 = 69x69 = 4761
tokens). MAX_HEIGHT_PX 1568->1932; dense tool/history pages now render at
DENSE_CONTENT_COLS=384 / DENSE_CONTENT_CHARS_PER_IMAGE=92160 (1928x1928 full
page) -- fewer image blocks at the same OCR-validated 5x8 cell. The static slab
is unchanged (313 cols / 1573x1280).
Also fixes a regression the code review caught: the break-even gate,
truncateForBudget, and estimateImageCount predicted against the slab geometry
(313 cols / 159 rows) while the dense renderer emits 384 cols / 240 rows, so
large tool_results were truncated far earlier than the 10-image cap required --
silently dropping output that would have rendered. A per-page char budget is
threaded through the prediction chain (defaults to READABLE so the slab and unit
tests are byte-identical) plus a denseGateGeometry helper, so the gate prices the
same page the renderer actually produces.
- src/dashboard/fragments.ts: server-side HTML fragment renderers reusing the
same JSON payloads (HTML and JSON surfaces can't drift)
- node.ts/dashboard.ts: /fragments/<name> routes (header, session-summary,
recent, latest, sessions, stats, toggle) with htmx polling + Alpine for
local state; toggle is a POST-returning-fragment htmx swap
- vendored htmx 2.0.4 + Alpine 3.14.9 in src/dashboard/vendor.ts (offline,
single-file npm package, no CDN)
- deleted Svelte components/stores/bundle build step; dropped svelte/vite
devDeps; build is now just tsc + esbuild
- tests: +5 fragment contract tests (routing, toggle state, HTML escaping of
source text, 404) - 339 total
Replace the hand-rolled dashboard with a Svelte component tree (App +
Sessions / StatsTable / LatestPng / RecentRequests / StatsHeader /
Cleanup / CompressionToggle / ToastTray), move the API layer into
lib/, and split state into stores/.
Add a dedicated dashboard build (`build:dashboard-ui` + a separate
tsconfig.dashboard.json) so dashboard sources typecheck independently
of the proxy core, which previously broke the root `tsc` build.
The proxy used to expose ~20 behavior toggles (--no-compress,
--no-tools, --no-schemas, --no-reminders, --no-tool-results,
--history, --no-history, --history-keep-tail N, --history-min-prefix N,
--min-chars N, --min-reminder-chars N, --min-tool-result-chars N,
--cols N, --multi-col N, --no-track, --events-file PATH, --port N,
--upstream URL) plus matching env vars. Each one was a dead branch in
the common case and a configuration knob nobody had a principled
reason to pick a value for.
Replaced with: exactly one way to run the proxy. Every compression
mode on, every tuning parameter at its measured-best value, history
compression always active (was opt-in dead code). The only adjustable
surface is deployment concerns — where to listen, what to forward,
where to log — env-var only:
PORT default 47821
ANTHROPIC_UPSTREAM default https://api.anthropic.com
PIXELPIPE_LOG default ~/.pixelpipe/events.jsonl
CLI accepts only --help and --version. Anything else exits with
non-zero and a one-line error.
Notable behavior change:
- DEFAULTS.compressHistory flipped false → true. Variant-C history-
image compression now runs on every request (was opt-in via
--history / COMPRESS_HISTORY=1). The static-slab cache_control
placement is on the system image, not on the history-replacement
chain, so the published cache-topology risk does not apply.
Touched:
- src/node.ts: CliOpts → RuntimeConfig (3 fields). parseCli is now
~25 lines. printHelp reduced to env-var-only doc.
- src/core/transform.ts: DEFAULTS.compressHistory = true. Comment
updated.
- scripts/restart.sh: drops PROXY_ARGS array entirely. Port read
from PORT= env var. Unknown args rejected.
- tests/restart.test.sh: replaces flag-passthrough test with a
reject-unknown-args test. 4/4 pass.
- tests/history.test.ts: renames the "compressHistory:false (default)"
test to reflect the new always-on default; asserts the input-not-
mutated invariant remains.
- README.md: configuration table reduced to 3 env vars; restart
examples drop CLI flag passthrough.
Tests: 277/277. Typecheck + build clean. Restart script tests: 4/4.
Unifont glyphs are inherently 1-bit (every pixel is 0 or 255). Storing as
8-bit bytes wasted 7 bits per pixel.
Changes:
- scripts/gen-atlas.ts: emit ATLAS_PIXELS as bit-packed MSB-first. OFFSETS
now indexes bits not bytes. Per-cell paint thresholds R-channel at 128
defensively (Unifont is already 0/255).
- src/core/render.ts:blitGlyph: byte read replaced with bit extraction
(~3 extra ops per pixel; negligible). max() blending removed since bit
data is binary, no AA edges, no glyph overlap.
- src/core/atlas.ts: regenerated with the new format. Header comment
documents the bit-extraction formula.
- tests/render.test.ts: +1 structural assertion that ATLAS_PIXELS.byteLength
matches the bit-packed total.
Bundle size:
atlas.ts raw: 5.47 MB → 1.06 MB (5.2× shrink)
atlas.ts gzip: 1.55 MB → 479 KB (3.2× shrink)
dist gzipped: 1.57 MB → 488 KB (3.15× shrink, well under Workers
free-tier 1 MB cap)
Pre-deflate shrink is 8× (theoretical max). The 3.15× post-deflate
result reflects that deflate was already exploiting long runs of 0/255
in the 8-bit data; the bit-packed bytes are denser and less compressible.
Render output is byte-identical to pre-bit-pack (storage format change
only). 108/108 vitest, 4/4 shell tests, typecheck + build clean.
- Switch font: JetBrains Mono @ 15px → Unifont 16.0.04 @ 10px. Sparse
atlas with double-width EAW dispatch for CJK. Full BMP profile
(35,501 glyphs) covers Latin/Cyrillic/Greek/Hebrew/Arabic/CJK/Hangul
plus Letterlike, Misc Technical, Block Elements, Geometric Shapes,
Misc Symbols, Dingbats, Enclosed Alphanumerics, all box-drawing.
- Fix cell-height bug (was hardcoded 9; now probed from font metrics
via Math.ceil(maxAscent+maxDescent)). Previously clipped the top
2 rows off every CJK glyph since the original Unifont ship.
- Expand tabs to spaces with 4-column tab stops, EAW-aware. Was the
silent-drop source for 99.6% of dropped chars in production.
- Add dropped_codepoints_top telemetry (top 20 per event, only emitted
when drops > 0) so future coverage gaps surface in events.jsonl.
- Raise minToolResultChars 2000→5000 and minReminderChars 1000→2000
to stop net-losing tokens on small compressions, per ~2,500 tok/img
empirical measurement (N=33 cold-miss events).
- Fix dashboard per-image-cost formula: was pngBytes/375 (~190 tok),
now imageCount * 2500 (the measured empirical rate). Reduces the
inflated "tokens saved" headline to honest numbers.
- Schema-stub fix in compressSchemas path: preserveSchemaShell()
retains type / properties / required / enum / oneOf / anyOf / $ref /
numeric constraints recursively (depth 20), strips only descriptions
and metadata. Prevents 400 errors on first-tool-call requests.
- Capture full gzipped 4xx body + per-event req_body_sha8 for upstream
error diagnosis. Sidecar to ~/.pixelpipe/4xx-bodies/ when over 32 KB
inline cap. Workers-safe via CompressionStream + Web Crypto.
- scripts/restart.sh: pnpm-aware one-command kill+rebuild+start with
graceful SIGTERM→SIGKILL escalation, build-failure abort, port-in-use
precheck, --no-build flag for fast iteration. Shell-tested.
- Bundle 1.58 MB gzipped on Node. 91/91 vitest + 4/4 shell tests.
Project rename for the Show HN launch with framing:
'pixelpipe — what if Opus saw a UI instead of just append-only logs?'
Renamed everywhere:
- kebab-case 'claude-image-proxy' -> 'pixelpipe' (42 occurrences)
- snake_case 'claude_image_proxy' -> 'pixelpipe' (5 occurrences)
- 14 files touched: package.json, Cargo.toml, Cargo.lock, *.md,
bin/cli.js, scripts/install.js, src/proxy.py, src/rust/src/main.rs,
src/rust/examples/{render_sample,transform_only,diff_render}.rs,
src/zig/build.zig
Effects on build artifacts:
- npm package name -> 'pixelpipe'
- Rust binary name -> target/release/pixelpipe (was claude-image-proxy)
- Python cache dir -> ~/.cache/pixelpipe/ (was ~/.cache/claude-image-proxy/)
NOT YET DONE in this commit (Tier 2+3, handled in next commit):
- Repo on-disk directory still ~/Downloads/repos/claude-image-proxy/
- Cache dir on disk still ~/.cache/claude-image-proxy/ — Python proxy
currently writes here. Will be migrated alongside the running services
in a coordinated stop/move/restart sequence.
Validation: scripts/diff_transform.py still produces exactly 1 structural
diff (the PNG bytes, expected — different fonts Python/Menlo vs
Rust/JetBrains Mono). No regression from the rename.
Three bug fixes surfaced by scripts/diff_transform.py harness, which
synthesizes a realistic Claude Code request body and runs it through both
Python transform_request() and Rust transform(), then JSON-structurally
diffs the outputs.
Before these fixes, harness found:
- 2 structural diffs in the transformed body JSON
- 4+ TransformInfo divergences
After these fixes, harness finds:
- 1 structural diff: $.messages[0].content[1].source.data
(the base64 PNG, expected to differ — Menlo vs JetBrains Mono fonts)
- TransformInfo still differs on byte-vs-char counts (telemetry-only)
Fixes:
1. src/rust/src/transform.rs:93 — per-tool markdown header
was: '### Tool: {name}\n{desc}'
now: '#### Tool: {name}\n{desc}\n'
Matches proxy.py:230,233. Also restores the trailing newline that
Python emits between tool entries.
2. src/rust/src/transform.rs:123 — tool section markdown header
was: '\n\n# Tool Documentation\n\n'
now: '\n\n#### Tool Documentation\n\n'
Matches proxy.py:248.
3. src/rust/src/transform.rs:265 — preserve system field by default
was: always replaced system with billing_line OR remainder
now: only replaces if billing_line was stripped, otherwise leaves
the original system field untouched.
Matches proxy.py:417 which intentionally leaves system alone in the
default PLACEMENT="user" path 'so Anthropic still has guardrails'.
The image lives in messages[].content as a side-channel; system stays
as original text. This is the behavior our 95+ live Python requests
have validated against Anthropic's API.
4. src/rust/src/lib.rs — expose 'transform' module so harnesses and the
transform_only example can call transform() without going through HTTP.
Added:
- src/rust/examples/transform_only.rs — CLI binary that reads a body
from a file and writes the transformed body + TransformInfo as JSON.
- scripts/diff_transform.py — Python harness that calls both Python
transform_request() and the Rust transform_only binary, then runs a
structural JSON diff with field-by-field TransformInfo comparison.
Validation:
python3 scripts/diff_transform.py
# → 'found 1 structural diff'
# → $.messages[0].content[1].source.data (the PNG, expected)
Remaining known telemetry-only diffs (not blocking the swap):
- system_text_chars: Python counts Unicode chars, Rust counts UTF-8 bytes
- tool_text_added: same UTF-8 issue
- expected_image_tokens: ~30% higher in Rust due to ab_glyph 4px cell
vs PIL 3px cell — Rust images are 33% wider for the same text. Rust
PNG bytes are still smaller overall because of the tight-bound height
(which Python also has now, after the previous baseline commit).
- placement, png_sha8 fields missing from Rust TransformInfo
- min_chars threshold uses bytes (Rust) vs chars (Python)
These can be addressed later for full telemetry parity but the
upstream-visible behavior is byte-equivalent.
src/proxy.py:125 was computing image height as
tallest = max(lines_per_col, last_col_lines) if c_needed == 1 else lines_per_col
The max(lines_per_col, ...) made every single-column render pad to the
full MAX_EDGE=1568 canvas height regardless of content, wasting ~99% of
the image-token budget for small prompts.
Fix: tight-bound the single-column case.
tallest = max(1, last_col_lines) if c_needed == 1 else lines_per_col
Multi-column (c_needed > 1) is unchanged — all earlier columns must be
lines_per_col tall to fit them, so the image must be too.
Measured impact (scripts/visual_compare.py):
sample before after change
----------- ------------ ------------ -----------
17 chars 241x1568 241x14 -99% pixels
176 chars 241x1568 241x42 -97% pixels
885 chars 241x1568 241x315 -80% pixels
Verified edge cases hold:
- empty/short text: still gated by MIN_COMPRESS_CHARS
- 5000-char single line: 241x441 (63 wrapped lines, tight)
- 80-char boundary: 241x14 (2 lines, tight)
- 500-line multi-column: 731x1568 (unchanged, correctly padded)
- 20KB realistic system prompt: 68.5% savings (matches existing 65-73%)
Added scripts/visual_compare.py to inspect Python vs Rust renderings
side-by-side with labels and dimensions.
This is the immediate win that's independent of the Rust rewrite — every
proxied request gets fewer image tokens starting at the next proxy restart.
Functional reimplementation in Rust with the same render_chunks/transform
public surface as src/proxy.py. Compiles clean on darwin-arm64, smoke-
tested but NOT YET battle-tested against real Claude Code sessions.
Layout:
src/rust/
Cargo.toml hyper + tokio + reqwest + ab_glyph + png + clap
Cargo.lock committed (application crate, deterministic builds)
src/main.rs clap CLI matching bin/cli.js flags
src/lib.rs module roots
src/font.rs ab_glyph rasterization of bundled JetBrains Mono
src/render.rs newspaper-column text -> PNG (matches Python algo)
src/transform.rs request body transformation (system prompt -> image)
src/proxy.rs hyper HTTP server, upstream forwarding
src/stats.rs persisted savings counters mirroring proxy.py
assets/JetBrainsMono-Regular.ttf OFL-licensed, legally embeddable
examples/render_sample.rs smoke test: render fibonacci snippet
examples/diff_render.rs diff harness companion (paired with
scripts/diff_render.py)
Diff harness:
scripts/diff_render.py feeds same text through Python and Rust
render_chunks, decodes both PNGs, compares
pixel-by-pixel. First run found two divergences:
1. Rust font.rs:43 uses ab_glyph h_advance which returns ~3.4px ceiled
to 4 for Menlo-5pt 'M'; PIL's getlength('M') returns 3.0 exactly.
Every Rust column is 33% wider than Python's.
2. Python proxy.py:125 pads every single-column image to MAX_EDGE=1568
height regardless of content (already noted in baseline commit).
Neither is fixed yet — surfacing the divergences before any swap. Next:
visual inspect both PNGs to confirm Rust output is at least OCR-readable,
then decide whether to fix bug-for-bug compat or correct both sides.
bin/cli.js still spawns Python; this commit only adds the Rust tree, it
does NOT change runtime behavior. The live proxy at :47821 is unaffected.
Token-saving HTTP proxy for Claude Code that renders system prompt + tools
as 5pt Menlo monospace images. Achieves 65-73% token reduction with 100%
reasoning quality preserved on Opus 4.7.
Production state at this commit:
- src/proxy.py: 972 lines, battle-tested with 46+ real Claude Code sessions
observed (~551k tokens saved, ~$8.27 in stats.json as of commit time)
- bin/cli.js: Node wrapper that spawns Python proxy on port 47821
- scripts/gen_atlas.py: generates the bundled Menlo 5pt glyph atlas
- scripts/install.js: postinstall — verifies Python + Pillow + httpx
- src/zig/: earlier Zig 0.16 renderer (kept for reference, not built)
- CLAUDE.md, HANDOFF.md: contributor notes
- package.json: npm entry point, version 0.1.0
Known issue (deferred for separate fix):
src/proxy.py:125 — `tallest = max(lines_per_col, last_col_lines) if
c_needed == 1 else lines_per_col` pads every single-column image to
full MAX_EDGE=1568 height regardless of content, wasting image-token
budget for small inputs. Tight bounding would improve savings further.
This is the rollback point. Any subsequent changes (Rust port, Python
fixes, etc.) build on this proven baseline.
Steven