Document that savings track uncached bulk the client still re-sends, not
the product name: Codex on /v1/responses is supported; warm sessions with
~98% cached_tokens honestly show low Saved until history collapse fires.
Document the Grok contract: opt-in only; when enabled, production 5x8
plus factsheet rather than looser cells. Point at the eval receipts and
how to read path / cached share / history_reason on the dashboard.
Grok was imaging on the Responses path with no readability data. Add
eval/grok-density: density ladder, climb receipts, and a live
image-vs-factsheet comparison on grok-4.5.
Findings locked into the receipts:
- Production 5x8 image-only confabulates exact IDs (0/4).
- Image + factsheet clears 4/4 exact, 0 confab, gist+guard ok (~70%
fixture savings).
- Looser pure-image packing can clear exact recall but only ~30%
savings — not worth the default.
Supports keeping Grok opt-in with dense 5x8 + factsheet when enabled.
Several families share /v1/responses. Cost, geometry, cache rates, and
whether imaging is even allowed must follow the model that serves the
request. Otherwise Claude and Grok inherit GPT defaults, the gate lies,
and the dashboard reports nonsense.
Claude on Responses:
- Bill images by Anthropic pixel area; cache 0.1x, output 5x.
- Use Anthropic page geometry (312 cols x 728 px). GPT height overstated
image cost ~2.6x and flipped every Opus slab to not_profitable, so
enabled Claude stayed text-only with blank As text / Saved.
History profitability gate:
- Reflow packs hard newlines with an inline ↵ glyph. countVisualRows
still treated ↵ as a row break, so estimateImageCount overstated pages
~6x on reflowed history and collapsed Claude/Grok history never fired.
Match the renderer: only hard newlines start a visual row.
Grok:
- Keep production 5x8 packing and a verbatim fact-sheet for OCR-hard
tokens (paths/hex/ports/camelCase). Measured ~1000 image tok/MPix;
cache 0.25x, output 3x.
- Leave Grok out of DEFAULT_MODEL_BASES (opt-in only, same bar as Opus).
Pure-image exact OCR fails at 5x8; do not image it silently.
Also thread optional per-model render style through the Responses path
and allow eval-only atlas overrides in gen-atlas.
- FINDINGS.md 2026-07-05 entry: capacity argument (misreads are
resolution-bound, receipts from the 2026-06-16 sweep), proposal-by-
proposal rejection table, and three documented follow-up threads
(glyph-style A/B with banked pages, runtime canary + re-fetch,
surrogate-reader pre-flight) for successors.
- docs/NOT-OCR.md: how VLM vision reads renders (patches, not glyphs),
why misses are silent confabulations, and why DeepSeek-OCR-era
skepticism predates the Opus 4.8 to Fable 5 capability jump.
- README: Roadmap now points at the parked state; two new FAQ entries
link to the doc instead of inlining it.
- Wording states observables only: token cost scales with pixel area,
dense text carries more information per token, and computer use
already ships models reading screenshots. No arbitrage/exploit
framing, no claims about pricing intent.
Ran eval/opus-density/run.mjs against the API (harness shipped without committed
results). Issue #6 answer: Opus 4.8 exact-string recall rises monotonically as the
render gets less dense — 1/4 -> 3/4 -> 4/4 across 5x8/7x10/9x12, reaching 4/4 exact
with 0 confabulations at 9x12 (45% savings), byte-exact hex, stop_reason end_turn.
Density is the lever; not a DEFAULT_MODEL_BASES change.
Two response-reading bugs fixed (neither involves images):
- callModel read content[0].text, but always-on-thinking models put a thinking
block with empty text at content[0]; the answer is in a later block. Every Fable
answer read as "" and scored miss/confab. Fixed: select the text block;
max_tokens 128 -> 512.
- score() ignored stop_reason, so a refusal (HTTP 200, stop_reason "refusal",
empty content) scored as a confabulation, inverting the safety verdict. Fixed:
treat refusal as its own state; a refused guard is safe.
Adds results.json (live run), RESULTS.md (write-up), verify.mjs (confound-free
receipts for both fixes plus the guard-question caveat: the never-stated password
guard trips Fable's cyber classifier equally on text and image, n=6, so it is not
an imaging effect and is unmeasurable on Fable as written).
The workflow only checks out, builds, and tests — it never writes to the repo,
so scope GITHUB_TOKEN down to contents: read (GitHub's recommended least
privilege; the default token grant is broader than this job needs). Also add a
per-ref concurrency group so a new push to a PR cancels its superseded run.
No job/step logic changed; Node stays on 22 (current LTS).
Issue #6 asks whether Opus 4.8 reads exact strings reliably if the text is
rendered less dense (bigger cells / fewer chars per page). This adds a
measurement harness — no production code, no allowlist change.
- eval/opus-density/run.mjs renders one synthetic session at three densities
(5×8 production, 7×10, 9×12), each with cols dropped so the page stays ≤1568×728
(the linear-billing window), then — when ANTHROPIC_API_KEY is set — asks
claude-opus-4-8 and claude-fable-5 a fixed battery (exact hex / camelCase / path
/ port, a gist question, and a never-stated guard) and scores exact accuracy,
confabulation, abstention, gist, image tokens, and savings.
- eval/opus-density/README.md documents the matrix, metrics, and the acceptance
bar: no silent wrong exact strings, gist == text baseline, savings still
positive — otherwise Opus stays opt-in (never added to DEFAULT_MODEL_BASES here).
Dry-run verified (no key needed): 5×8→79%, 7×10→62%, 9×12→45% savings at
1568×{128,228,344} px. Results are not committed — run on a machine with API
access.
Broaden the guard from README + top-level docs/*.md inline links to the full
tracked doc set and more link forms:
- scan README.md, CHANGELOG.md, FINDINGS.md, docs/**/*.md, and eval/**/README.md
(eval run logs are skipped — only their READMEs are prose)
- parse inline links, image links, reference-style definitions, and simple HTML
href=/src=
- validate the target file exists AND, when a link carries a #fragment into a
markdown file, that a matching GitHub-style heading anchor exists
Verified: passes on the current tree and fails on an injected broken link or
broken #anchor. (A README public-API-example typecheck lands on the integration
branch, where the README import fix is present.)
A static audit can only catch broken doc links once; this keeps them honest on
every run. A dependency-light vitest test (pure fs + regex, no new deps, already
picked up by `pnpm test` and CI) walks README.md and docs/*.md, extracts each
relative link/image target (skipping http/mailto/data/# and stripping
fragments/queries), and asserts the target exists.
Verified: passes on the current tree (all 7 relative links resolve, including
docs/LEGIBILITY-AUDIT-2026-07-01.md) and fails with a clear per-link message
when a dead link is introduced.
Strengthen the >0 assertion into a structural one: with a rendered image, a
pointer, an input_text prompt, and a rewritten tool in the outgoing body, assert
outgoingTextChars (a) counts the instructions pointer + input_text parts + tool
name/description/parameters (>= their summed length, within the '\n\n' part
separators), and (b) stays well below the input_image base64 length — i.e. the
base64 is excluded from the regression denominator, as intended.
transformOpenAIChatCompletions set info.outgoingTextChars = countOutgoingTextChars(req)
before returning; transformOpenAIResponses set info.compressed = true and returned
without it. That field is the regression denominator (tokens ≈ α·outgoingTextChars +
β·imagePixels), so compressed Responses rows silently carried no denominator.
- add countResponsesOutgoingTextChars(): instructions + message-item text
(string or input_text parts, via responsesContentText so input_image base64 is
excluded) + flat tool name/description/parameters
- set it on the Responses success path, mirroring the Chat path
- test asserts outgoingTextChars > 0 for a compressed Responses request
shouldTransformAnthropicMessages() gated the path with
!path.endsWith('/v1/messages'), but createProxy() routes three exact paths:
/v1/messages, /anthropic/v1/messages, and /anthropic/messages. So an SDK/host
using the public helper to pre-gate got unsupported_path on /anthropic/messages
— a request the proxy actually transforms — while endsWith would also have
wrongly accepted an unrelated /foo/v1/messages.
- lift isAnthropicMessagesPath() into applicability.ts (proxy.ts already imports
from it; no cycle) as the single source of truth
- proxy.ts imports it instead of its own copy
- shouldTransformAnthropicMessages() uses it, so helper and router agree
- public-api tests cover /anthropic/v1/messages and /anthropic/messages
(eligible) plus /v1/messages/count_tokens (still unsupported_path)
The exclude comment described src/dashboard as "the Svelte browser bundle —
compiled separately by scripts/build-dashboard-ui.mjs ... covered by
tsconfig.dashboard.json for editor + CI type-checking." None of that is true:
- src/dashboard/ is server-rendered HTML (htmx + Alpine, both inlined by
scripts/vendor-ui.mjs) — not Svelte, and there is no esbuild-svelte step
- scripts/build-dashboard-ui.mjs and tsconfig.dashboard.json do not exist
- CI (.github/workflows/ci.yml) runs only the root typecheck/test/build
The exclude itself was a no-op: src/dashboard.ts (which IS in the program)
imports ./dashboard/fragments.js and ./dashboard/types.js, so tsc already
type-checks and emits dist/dashboard/* transitively. Verified: removing the
exclude leaves typecheck green and dist/dashboard/* emits identically with or
without it. Dropped both the false comment and the dead exclude entry.
The Responses API allows a message item's content to be a string OR an array of
input_text parts. transformOpenAIResponses only read the string form
(`typeof content === 'string' ? content : ''`), so a developer/system item sent
as [{ type: 'input_text', text: ... }] was neither imaged nor stubbed — its
verbose text rode uncompressed as native input, and the pointer replacement
(also string-only) left it in place.
- collect static context via the existing responsesContentText() helper, which
reads both shapes
- pointer replacement now handles arrays too, preserving the array shape
([{ type: 'input_text', text: RESPONSES_POINTER }]) so a parts-form request is
not reshaped into a string
- test: a developer item with array content is imaged (staticChars counts it)
and its outgoing content is a single pointer part, no original text
The original bug lived in collectSource's --git untracked branch, not in the
readExportTextFile helper the unit tests cover. Add an E2E test that runs the
actual CLI (tsx src/node.ts export --git --include '*.ts') against a throwaway
git repo with a tracked baseline plus untracked keep.ts, a 5000-char skip.md, an
oversized huge.ts, and a binary bin.ts. Asserts oversized + binary untracked
files are skipped (with warnings) and that only keep.ts's content reaches the
source (sourceChars < 500 — skip.md would have added thousands).
In --git mode, untracked files (git ls-files --others) were read straight to
memory with only a binary check — unlike directory and single-file modes, which
also apply --include/--exclude globs and the 1 MiB MAX_FILE_BYTES cap. So
`pxpipe export --git --include '*.ts'` still slurped every untracked file
regardless of the filter, and an untracked multi-MB file was read whole (a
resource-safety hazard).
- extract the shared gate into src/export-collect.ts: readExportTextFile()
applies include/exclude → size → binary, in that order, and is import-safe
(src/core/export.ts is contractually fs-free, so it can't live there)
- route all three collection paths (walkDir, single-file target, --git
untracked) through it; untracked skips now warn like explicit targets do
- tests/export-collect.test.ts covers include, exclude, oversized, the size
boundary, binary, and inaccessible
Verified end-to-end: `export --git --include '*.ts'` in a temp repo excludes an
untracked .md, skips an oversized .ts (with a warning), and keeps the .ts.
printVersion() logged `process.env.npm_package_version ?? '0.2.0'`. npm only
populates npm_package_version inside its own run-script env, so an end user
running `npx pxpipe-proxy --version` or a globally-installed `pxpipe` got the
stale hardcoded fallback (0.2.0) instead of the real version — the code comment
even claimed an esbuild.define that was never wired up.
- scripts/build.mjs reads package.json and inlines the version via esbuild
`define: { __PXPIPE_VERSION__ }`, then smoke-checks the built binary
(`node dist/node.js --version` must equal package.json) and fails the build
on mismatch, so a broken injection can never ship.
- src/node.ts reads the injected constant behind a `typeof` guard (safe under
tsx, where it is undefined), falling back to npm_package_version then
'unknown' — never a stale release number.
Verified: node dist/node.js --version now prints 0.8.0.
The 'Library use' snippet imported `renderTextToPngs` from "pxpipe", but the
published package is "pxpipe-proxy" and the root barrel (src/core/index.ts)
exports `renderTextToImages`, not `renderTextToPngs`. Both the import
specifier and the function name were unrunnable as written.
- import from "pxpipe-proxy" (matches package.json "name")
- use renderTextToImages, which returns { pages, droppedChars, pixels }
- show pages[i].png (Uint8Array) instead of the stale RenderedImage[] comment
If ANTHROPIC_API_KEY / OPENAI_API_KEY is configured on a deployed
Worker, the URL becomes an open key-spender: anyone who discovers it
gets requests signed with the deployment's key. Now:
- keys set + no PXPIPE_WORKER_SECRET -> 503, refuses to proxy at all
- keys set + secret -> callers must send x-pxpipe-secret (compared via
SHA-256 digests to avoid a prefix-timing signal); 401 otherwise
- no keys set -> unchanged (callers bring their own credentials)
The secret is stripped before forwarding upstream.
The dashboard HTML is served from the same origin as the JSON API, so
same-origin fetches never consult CORS. The wildcard header only served
cross-origin readers - i.e. any webpage open in the operator's browser
could read captured context out of 127.0.0.1. Browser default (opaque
cross-origin responses) is exactly what we want here.
The Node proxy called server.listen(port) with no host, so it bound all
interfaces (0.0.0.0/::) while the startup log printed http://127.0.0.1.
The dashboard is unauthenticated and serves captured request context
(/api/image-source returns the source text rendered into images) plus a
compression kill switch, with access-control-allow-origin: *. On a shared
network that context was readable by anyone who could reach the port.
Bind to 127.0.0.1 by default; add a HOST env to explicitly opt into
all-interfaces exposure (container/host-access use cases), with a startup
warning when bound off-loopback. IPv6 handled: ::1/localhost count as
loopback (no warning) and the startup URL is bracket-formatted. The one
README client example that used localhost is pinned to 127.0.0.1 to match
the IPv4 loopback default on IPv6-first hosts.
Verified: default refuses off-host connections; HOST=0.0.0.0 restores the
previous behavior + warns; HOST=::1 binds IPv6 loopback with a valid
bracketed URL and no warning. typecheck + build pass.
- staleFreshnessHints() rewrites the upstream "(file state is current in
your context - no need to Read it back)" hint into a stale-state warning
at history-serialization time, covering both collapsed same-session
turns and slabs inherited by continuation sessions
- Edit/Write/NotebookEdit stubs gain a live-text same-session Read
precondition (READ_FIRST_TOOLS), riding un-imaged in the tools array
- telemetry: churning_static_tags canary - FNV-1a fingerprint per
session/tag flags slab tags whose content churns and busts the image
cache (TAG_OBSERVATIONS_MAX-capped)
- 5 regression tests in tests/history.test.ts; baseline to beat: 97
read-gate errors across 18 sessions
Opening-turn task prompts were reduced to a 300-char compactPreview
tombstone by demoteProtectedHeadText, losing questions and trailing
output-format instructions that no later turn restated. The tombstone
keeps its cheap cache-stable preview, but latestCollapsedUserPointer
now scans INCLUDING the protected head: when the opening turn is the
only typed user text in the collapsed range (single-task sessions),
the pointer carries it verbatim via verbatimTaskText — up to 4000
chars, head 2600 / tail 1400 elision beyond that.
- README: Fable clip as headline demo (Drive link + committed thumb),
verified numbers from the recording: plain $42.21 / 96% context vs
pxpipe $4.51; honest caveat on single-reply format compliance
- ATTEMPTS.md: attempt 1 (invalid run, model-switch pitfall) and
attempt 2 (legibility PASS on fable, format miss documented)
- a.sh/b.sh: prompt fix — numbering starts at filler-000
Volatile env/context text (git status, cwd, date) rode in req.system —
BEFORE the slab anchor in Anthropic's prefix order (tools → system →
messages) — so any git-state change cold-restarted the entire anchored
prefix. Telemetry attribution (events.jsonl 2026-06-26..07-02): 48.8%
of cold-create waste, ~2.6k tokens/session vs ~200 saved by imaging it.
Now appended as a trailing text block on the LAST user message — the
per-turn live tail that re-caches incrementally anyway — placed AFTER
history collapse (never baked into frozen chunks) and after tool_result
compression. Session-stable billingLine/sysRemainder stay in system;
fallback keeps env in system when no user message exists to carry it.
New telemetry field: envRelocatedChars.
The Chrome extension attaches over its own channel (--strict-mcp-config
does not cover it) and injects tools mid-session, mutating the tool
array and busting the prompt-cache prefix from that point on. This
landed asymmetrically between arms and inflated b.sh's cache-write
cost. With --no-chrome the tool surface is frozen from request 1 in
both arms; a fresh A/B pair confirmed: $1.26 control vs $0.78 pxpipe,
cache writes 46.8k vs 22.5k, no write fail, no safeguard trip.
Fix#1 (cross-session cache bust): strip the volatile "# Environment"
markdown section (working dir, git status, platform, model ID) out of
the system text BEFORE the static/dynamic split. It carries no XML
wrapper so splitStaticDynamic couldn't catch it, and its git-status
lines change across sessions - baking them into the slab PNG busted
the cross-session cache (system_sha8 717f1fce -> 5efaa4bb for a
one-file edit). Parallel to stripBillingLine: the section re-enters
the system tail as plain text.
Schema handling simplified, knob removed: the Anthropic text-reference
path now renders prose-only tool docs (schemas stay untouched in
tools[] - duplicating them in the reference was pure bloat), while the
GPT imaged path always carries the compact schema (there it IS the
compression). Deletes the {type:'object'} stub + SCHEMA_STRUCTURAL_KEYS
machinery, stripSchemaDescriptions import, and the COMPRESS_SCHEMAS env.
Tests: e2e regressions for anchor relocation onto byte-frozen carry-over
history images and first-collapse anchor staying on the slab; history
relocation fixture grown past the collapseChunk=50 snap (56 messages ->
cutoff 50, frozen windows 11/21/31/41, collapsedTurns 49 - the old
35-message fixture floored to one window and never froze a chunk).
tsc clean; 615/615 tests pass.
The text-based Tool Reference triggered Anthropic's model-cloning
safeguard (stop_reason: refusal -> silent Opus fallback) because the
per-tool stub said docs lived in "the system prompt" — the exact
phrasing the 169521c retrip identified as the trigger for the imaged
slab, now reintroduced in text form.
- Stub now reads: 'ⓘ Full docs: see "## Tool: <name>" in the Tool
Reference section.' — no "system prompt"/"authoritative" framing.
- Tool Reference header is provenance-framed as first-party ("pxpipe
(this user's local proxy) moved the full tool documentation...").
- tracker.ts now maps tool_docs_chars into events.jsonl so refusals
are attributable to the text-reference path.
- Regression test: stub + reference header must never match
/system prompt|authoritative/i and must carry provenance framing
(scoped to the header; quoted third-party tool docs below it may
legitimately contain the phrase).
Verified: 614/614 tests, tsc --noEmit clean, and a live cold-start
through the rebuilt proxy on claude-fable-5 with the text reference
active — no refusal event, no fallback.
The in-image banner announced 'SYSTEM PROMPT + TOOL DOCS ... treat as
authoritative system instructions' inside a user turn. Anthropic's
model-cloning classifier read that as a replayed/extracted prompt and
refused (apiRefusalCategory: reasoning_extraction), forcing Claude Code
to fall back to claude-opus-4-8 — outside compress scope, so the whole
session ran passthrough with zero savings.
Reworded to first-party provenance framing (pxpipe rendered this
session's own configuration; no 'system prompt'/'authoritative').
Two headless probes through :47824 now serve on claude-fable-5[1m]
with the imaged slab (img:4, ~94k chars) and zero refusal events;
identical shape was refused pre-fix (req_011CccRnutLzRJNVZ7bRKG8Q).
Anthropic's safety classifier OCRs rendered images. A *legible* image of the
system prompt + tool docs (bash / permissions / credential / file-write
vocabulary) trips its prompt-injection heuristic and forces a model downgrade
to Opus, which costs far more than imaging the (already prompt-cached) slab
ever saves. Empirically the caption wording is NOT the trigger — the OCR'd
content is — so no caption rewrite fixes it; the slab must stay text.
Gate at the slab profitability check with PXPIPE_TEXT_SLAB=1. Default is
unchanged (legacy imaged slab) so the cache-align / cache-stability suites
stay valid. Both demos now launch the pxpipe proxy with the toggle set.
Verified: identical ~300KB-slab request images 8 PNGs by default, 0 with the
toggle (reason=slab_text). Full suite: 614 passed.
Both demos' a.sh/b.sh now request claude-fable-5[1m] for the default `fable`
arm, consistent with the opus/sonnet arms which already use [1m].
setup.sh PXPIPE_MODELS stays the BASE claude-fable-5 on purpose: the proxy
strips [variant] tags from the incoming model before matching against the
allow-list verbatim (src/core/applicability.ts), so base claude-fable-5
already covers claude-fable-5[1m]. Writing [1m] into the allow-list would
make the stripped incoming base no longer match -> pxpipe silently stops
compressing. Added a comment in each setup.sh pinning that.
Measured 2026-07-01 (count_tokens sweep, claude-sonnet-4-5; see
docs/LEGIBILITY-AUDIT-2026-07-01.md): the API downscales every image to fit
BOTH long-edge ≤1568 AND ~1.15 MP (≈1,143,750 px), then bills ≈px/750
(~1525 tok/img cap). The old 1932×1932 page hit the cap but was resampled
0.555× before the vision encoder, so 5×8 glyphs arrived at ~2.8×4.4 px — the
root cause of the legibility misses this session.
New page shape 1568×728 = 1,141,504 px fits both bounds → glyphs reach the
encoder WYSIWYG (and still ≤2000 px/side for >20-image requests).
render.ts geometry:
MAX_HEIGHT_PX 1932 -> 728
MAX_WIDTH_PX 1932 -> 1568
DEFAULT_COLS 313 -> 312 (1568 px exactly; 313=1573 would 0.997× blur)
DENSE_CONTENT_COLS 384 -> 312
READABLE/DENSE chars 50000/92160 -> 28080 (312×90, tracks real pagination)
Decouple the GPT path: OpenAI resizes differently (2048 bbox, 768 short side),
so a 768-wide strip up to 2048 tall survives un-resampled. Add
GPT_MAX_HEIGHT_PX = 1932 (gpt-model-profiles.ts) where the built-in cost
numbers were calibrated; openai-history.ts now uses it instead of the
now-Anthropic-specific MAX_HEIGHT_PX. Doc-only touch-ups in library.ts/export.ts.
tests/render + tests/paging updated to the new geometry (171 pass).
Steven Chong