Keep non-cache-controlled appended system text in the Anthropic system field when the cacheable system slab is rendered to images.
Generated by Ora Studio
Vibe coded by ousamabenyounes
- docs-integrity: normalize walker paths to forward slashes so keys match on Windows
- export-git-e2e: run tsx's JS entry via node instead of the .bin/.cmd shim (same pattern as #63)
- vitest: 30s testTimeout for CPU-bound PNG-render tests on slower machines
(build.mjs portion of the original branch dropped — superseded by #63)
scripts/build.mjs spawned `pnpm exec tsc` with `spawnSync('pnpm', ...,
{ shell: false })`. On Windows the launcher is `pnpm.cmd`, which spawnSync
cannot run without a shell: it returns `status: null` (ENOENT), so the
`process.exit(tsc.status ?? 1)` line exited the build with code 1 and no
output — `pnpm run build` failed silently.
Run tsc's JS entry with the current Node binary instead
(`node <resolved typescript/bin/tsc>`), mirroring the existing --version
smoke check. This avoids the .cmd spawn problem (and the shell:true DEP0190
arg-escaping warning) entirely, is cross-platform, and removes the build's
assumption that pnpm is the caller. Also surface spawn errors explicitly
instead of collapsing them into a bare exit 1.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The model id is interpolated into the single-quoted hx-vals attribute. A
crafted id (reachable as an `active` model from any /v1/messages request on
the unauthenticated localhost dashboard) could break out of the JSON or the
attribute itself. Build valid JSON via JSON.stringify, then escapeHtml the
whole attribute value; the browser decodes the entities before htmx reads it,
so well-formed ids are unaffected. Closes#58.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Request bodies hold full prompts and any secrets in context. On 4xx (429s are
routine) they were gzipped to ~/.pxpipe/events.jsonl / 4xx-bodies/ with no
retention limit. Gate the capture behind captureErrorReqBody (env
PXPIPE_DEBUG_CAPTURE_4XX=1), default off, with a one-time startup warning. The
upstream errorBody (no user content) and the sha8 hash still land. Closes#69.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Server-side tools (e.g. advisor_20260301, web_search_20250305) were
getting a description injected and a heading in the imaged Tool
Reference, causing Anthropic to 400 with "Extra inputs are not
permitted". Guard the rewrite map so any tool with a non-"custom"
type passes through byte-identical, mirroring the OpenAI-path
isFunctionTool guard.
Fixes#43
Claude Code marks its system prompt with cache_control
{type:"ephemeral", ttl:"1h", scope:"global"}. When pxpipe relocates
that marker onto a rendered image, the new position is never a valid
global prefix (slab pages 1..N-1 and other injected blocks carry no
marker), so Anthropic rejects the whole request with 400
invalid_request_error — making pxpipe unusable in front of Claude Code.
Demote relocated markers to plain ephemeral by stripping the scope key
(type/ttl preserved) at all four relocation sites: the slab->history
anchor move, the static-system marker, and the reminder / tool_result
image renders. Markers without scope pass through untouched, so
byte-stability of existing traffic is unaffected. pxpipe still never
adds markers of its own.
Closes#95
The rewrite dropped the implicit 'error' listener that events.once()
used to attach. If the ServerResponse emits 'error' while awaiting
drain, Node raises an unhandled 'error' event and crashes the process.
Attach a third once-listener and detach all three on whichever fires
first.
Found during maintainer review of #91.
waitForDrain raced Promise.race([once(out,'drain'), once(out,'close')]).
The losing once() listener was never removed, and events.once() also
attaches an implicit 'error' listener, so every backpressure cycle on a
long streamed response leaked one 'close' + one 'error' listener on the
same ServerResponse. Over long SSE streams this triggers
MaxListenersExceededWarning, unbounded heap growth, and eventually a
silent OOM exit of the proxy (observed as 'Connection closed mid-response'
on the client with no shutdown log).
Manage the two listeners manually and remove both on whichever fires
first. Behaviour is unchanged: drain resolves, close rejects with
'client response closed'.
Mainly ships the subagent env-relocation fix: the relocated environment
block was landing in the last user message every turn, steering Agent
subagent calls to fable instead of haiku. Diff-based static/volatile env
split (splitEnvByVolatility) fixes it. Also includes Sol opt-in default
removal, per-model render profiles, and dashboard total scoping already
in the Unreleased log.
The relocated '# Environment' block put model-identity/catalog lines in
the LAST user message every turn, steering subagent picks to fable
instead of haiku (LinkedIn report). splitEnvByVolatility now learns per
project (claudeMdSha) which env entries are byte-stable and promotes
them into the imaged slab; churned/new entries keep the live-tail
relocation, with the identity regex retained as a second layer.
Invariants (pinned by tests):
- first-ever sightings stay volatile: git state never bakes into a
fresh session's image (48.8% cold-create fix intact)
- the slab never re-renders mid-session: static side frozen byte-exact;
a churned promoted entry re-emits fresh text on the tail (supersedes
its stale slab copy) and demotes next session via sticky churn history
- no user message -> env stays in system, unchanged
Telemetry: envStaticChars / envVolatileKeys.
A (IDS rows + factsheet) 14/24, B (factsheet only) 14/24,
C (no guards) 8/24. Factsheet delivers the recall lift; IDS rows
show no measurable effect. Evidence for dropping the guards
paragraph from MODEL_RENDER_PROFILES.md.
Ship Grok 5×8 white/IDS packing and universal IDS/factsheet plumbing while
keeping DEFAULT_MODEL_BASES Fable-only: live pure-image Grok is not
Fable-level; image+factsheet is the working exact path.
Chart shows Grok as a text-window series only (no Grok+pxpipe overlay) and
Fable as the measured pxpipe density point. Add Codex-path quality harnesses
and receipts (multi-seed, fix-matrix, shipped smoke).
Plot characters-held per frontier context window, 2018-2026, ending at
the same Fable 5 1M window read through pxpipe images: a measured ~18M
chars vs the ~4M ceiling every text line converges on.
The chart is generated, not drawn: gen-context-chart.ts measures
chars/vision-token from a live pipeline render at generation time and
derives the 4.6x multiplier from that, so the claim regenerates with
the code it describes. Historical points carry their sourcing as code
comments — including the two a reviewer would check: Gemini 1.5 Pro is
plotted at its June 2024 2M peak (the line honestly dips to 2.5 Pro's
1M), and Sonnet 4 [1m] appears as Anthropic's first 1M window so the
Claude line doesn't imply Fable 5 was first.
The GPT tests drive gpt-5.6-sol, which is intentionally outside the
built-in default scope (Fable 5 only). Without pinning, the file's
result depended on the developer's ambient PXPIPE_MODELS — passing
locally but failing in CI's clean env. Snapshot/pin/restore, same
convention as proxy-usage.test.ts.
Document that savings track uncached bulk the client still re-sends, not
the product name: Codex on /v1/responses is supported; warm sessions with
~98% cached_tokens honestly show low Saved until history collapse fires.
Document the Grok contract: opt-in only; when enabled, production 5x8
plus factsheet rather than looser cells. Point at the eval receipts and
how to read path / cached share / history_reason on the dashboard.
Grok was imaging on the Responses path with no readability data. Add
eval/grok-density: density ladder, climb receipts, and a live
image-vs-factsheet comparison on grok-4.5.
Findings locked into the receipts:
- Production 5x8 image-only confabulates exact IDs (0/4).
- Image + factsheet clears 4/4 exact, 0 confab, gist+guard ok (~70%
fixture savings).
- Looser pure-image packing can clear exact recall but only ~30%
savings — not worth the default.
Supports keeping Grok opt-in with dense 5x8 + factsheet when enabled.
Several families share /v1/responses. Cost, geometry, cache rates, and
whether imaging is even allowed must follow the model that serves the
request. Otherwise Claude and Grok inherit GPT defaults, the gate lies,
and the dashboard reports nonsense.
Claude on Responses:
- Bill images by Anthropic pixel area; cache 0.1x, output 5x.
- Use Anthropic page geometry (312 cols x 728 px). GPT height overstated
image cost ~2.6x and flipped every Opus slab to not_profitable, so
enabled Claude stayed text-only with blank As text / Saved.
History profitability gate:
- Reflow packs hard newlines with an inline ↵ glyph. countVisualRows
still treated ↵ as a row break, so estimateImageCount overstated pages
~6x on reflowed history and collapsed Claude/Grok history never fired.
Match the renderer: only hard newlines start a visual row.
Grok:
- Keep production 5x8 packing and a verbatim fact-sheet for OCR-hard
tokens (paths/hex/ports/camelCase). Measured ~1000 image tok/MPix;
cache 0.25x, output 3x.
- Leave Grok out of DEFAULT_MODEL_BASES (opt-in only, same bar as Opus).
Pure-image exact OCR fails at 5x8; do not image it silently.
Also thread optional per-model render style through the Responses path
and allow eval-only atlas overrides in gen-atlas.
- FINDINGS.md 2026-07-05 entry: capacity argument (misreads are
resolution-bound, receipts from the 2026-06-16 sweep), proposal-by-
proposal rejection table, and three documented follow-up threads
(glyph-style A/B with banked pages, runtime canary + re-fetch,
surrogate-reader pre-flight) for successors.
- docs/NOT-OCR.md: how VLM vision reads renders (patches, not glyphs),
why misses are silent confabulations, and why DeepSeek-OCR-era
skepticism predates the Opus 4.8 to Fable 5 capability jump.
- README: Roadmap now points at the parked state; two new FAQ entries
link to the doc instead of inlining it.
- Wording states observables only: token cost scales with pixel area,
dense text carries more information per token, and computer use
already ships models reading screenshots. No arbitrage/exploit
framing, no claims about pricing intent.
Ran eval/opus-density/run.mjs against the API (harness shipped without committed
results). Issue #6 answer: Opus 4.8 exact-string recall rises monotonically as the
render gets less dense — 1/4 -> 3/4 -> 4/4 across 5x8/7x10/9x12, reaching 4/4 exact
with 0 confabulations at 9x12 (45% savings), byte-exact hex, stop_reason end_turn.
Density is the lever; not a DEFAULT_MODEL_BASES change.
Two response-reading bugs fixed (neither involves images):
- callModel read content[0].text, but always-on-thinking models put a thinking
block with empty text at content[0]; the answer is in a later block. Every Fable
answer read as "" and scored miss/confab. Fixed: select the text block;
max_tokens 128 -> 512.
- score() ignored stop_reason, so a refusal (HTTP 200, stop_reason "refusal",
empty content) scored as a confabulation, inverting the safety verdict. Fixed:
treat refusal as its own state; a refused guard is safe.
Adds results.json (live run), RESULTS.md (write-up), verify.mjs (confound-free
receipts for both fixes plus the guard-question caveat: the never-stated password
guard trips Fable's cyber classifier equally on text and image, n=6, so it is not
an imaging effect and is unmeasurable on Fable as written).
The workflow only checks out, builds, and tests — it never writes to the repo,
so scope GITHUB_TOKEN down to contents: read (GitHub's recommended least
privilege; the default token grant is broader than this job needs). Also add a
per-ref concurrency group so a new push to a PR cancels its superseded run.
No job/step logic changed; Node stays on 22 (current LTS).
Issue #6 asks whether Opus 4.8 reads exact strings reliably if the text is
rendered less dense (bigger cells / fewer chars per page). This adds a
measurement harness — no production code, no allowlist change.
- eval/opus-density/run.mjs renders one synthetic session at three densities
(5×8 production, 7×10, 9×12), each with cols dropped so the page stays ≤1568×728
(the linear-billing window), then — when ANTHROPIC_API_KEY is set — asks
claude-opus-4-8 and claude-fable-5 a fixed battery (exact hex / camelCase / path
/ port, a gist question, and a never-stated guard) and scores exact accuracy,
confabulation, abstention, gist, image tokens, and savings.
- eval/opus-density/README.md documents the matrix, metrics, and the acceptance
bar: no silent wrong exact strings, gist == text baseline, savings still
positive — otherwise Opus stays opt-in (never added to DEFAULT_MODEL_BASES here).
Dry-run verified (no key needed): 5×8→79%, 7×10→62%, 9×12→45% savings at
1568×{128,228,344} px. Results are not committed — run on a machine with API
access.
Broaden the guard from README + top-level docs/*.md inline links to the full
tracked doc set and more link forms:
- scan README.md, CHANGELOG.md, FINDINGS.md, docs/**/*.md, and eval/**/README.md
(eval run logs are skipped — only their READMEs are prose)
- parse inline links, image links, reference-style definitions, and simple HTML
href=/src=
- validate the target file exists AND, when a link carries a #fragment into a
markdown file, that a matching GitHub-style heading anchor exists
Verified: passes on the current tree and fails on an injected broken link or
broken #anchor. (A README public-API-example typecheck lands on the integration
branch, where the README import fix is present.)
A static audit can only catch broken doc links once; this keeps them honest on
every run. A dependency-light vitest test (pure fs + regex, no new deps, already
picked up by `pnpm test` and CI) walks README.md and docs/*.md, extracts each
relative link/image target (skipping http/mailto/data/# and stripping
fragments/queries), and asserts the target exists.
Verified: passes on the current tree (all 7 relative links resolve, including
docs/LEGIBILITY-AUDIT-2026-07-01.md) and fails with a clear per-link message
when a dead link is introduced.
Strengthen the >0 assertion into a structural one: with a rendered image, a
pointer, an input_text prompt, and a rewritten tool in the outgoing body, assert
outgoingTextChars (a) counts the instructions pointer + input_text parts + tool
name/description/parameters (>= their summed length, within the '\n\n' part
separators), and (b) stays well below the input_image base64 length — i.e. the
base64 is excluded from the regression denominator, as intended.
transformOpenAIChatCompletions set info.outgoingTextChars = countOutgoingTextChars(req)
before returning; transformOpenAIResponses set info.compressed = true and returned
without it. That field is the regression denominator (tokens ≈ α·outgoingTextChars +
β·imagePixels), so compressed Responses rows silently carried no denominator.
- add countResponsesOutgoingTextChars(): instructions + message-item text
(string or input_text parts, via responsesContentText so input_image base64 is
excluded) + flat tool name/description/parameters
- set it on the Responses success path, mirroring the Chat path
- test asserts outgoingTextChars > 0 for a compressed Responses request
Steven Chong