Broaden the guard from README + top-level docs/*.md inline links to the full
tracked doc set and more link forms:
- scan README.md, CHANGELOG.md, FINDINGS.md, docs/**/*.md, and eval/**/README.md
(eval run logs are skipped — only their READMEs are prose)
- parse inline links, image links, reference-style definitions, and simple HTML
href=/src=
- validate the target file exists AND, when a link carries a #fragment into a
markdown file, that a matching GitHub-style heading anchor exists
Verified: passes on the current tree and fails on an injected broken link or
broken #anchor. (A README public-API-example typecheck lands on the integration
branch, where the README import fix is present.)
A static audit can only catch broken doc links once; this keeps them honest on
every run. A dependency-light vitest test (pure fs + regex, no new deps, already
picked up by `pnpm test` and CI) walks README.md and docs/*.md, extracts each
relative link/image target (skipping http/mailto/data/# and stripping
fragments/queries), and asserts the target exists.
Verified: passes on the current tree (all 7 relative links resolve, including
docs/LEGIBILITY-AUDIT-2026-07-01.md) and fails with a clear per-link message
when a dead link is introduced.
Strengthen the >0 assertion into a structural one: with a rendered image, a
pointer, an input_text prompt, and a rewritten tool in the outgoing body, assert
outgoingTextChars (a) counts the instructions pointer + input_text parts + tool
name/description/parameters (>= their summed length, within the '\n\n' part
separators), and (b) stays well below the input_image base64 length — i.e. the
base64 is excluded from the regression denominator, as intended.
transformOpenAIChatCompletions set info.outgoingTextChars = countOutgoingTextChars(req)
before returning; transformOpenAIResponses set info.compressed = true and returned
without it. That field is the regression denominator (tokens ≈ α·outgoingTextChars +
β·imagePixels), so compressed Responses rows silently carried no denominator.
- add countResponsesOutgoingTextChars(): instructions + message-item text
(string or input_text parts, via responsesContentText so input_image base64 is
excluded) + flat tool name/description/parameters
- set it on the Responses success path, mirroring the Chat path
- test asserts outgoingTextChars > 0 for a compressed Responses request
shouldTransformAnthropicMessages() gated the path with
!path.endsWith('/v1/messages'), but createProxy() routes three exact paths:
/v1/messages, /anthropic/v1/messages, and /anthropic/messages. So an SDK/host
using the public helper to pre-gate got unsupported_path on /anthropic/messages
— a request the proxy actually transforms — while endsWith would also have
wrongly accepted an unrelated /foo/v1/messages.
- lift isAnthropicMessagesPath() into applicability.ts (proxy.ts already imports
from it; no cycle) as the single source of truth
- proxy.ts imports it instead of its own copy
- shouldTransformAnthropicMessages() uses it, so helper and router agree
- public-api tests cover /anthropic/v1/messages and /anthropic/messages
(eligible) plus /v1/messages/count_tokens (still unsupported_path)
The exclude comment described src/dashboard as "the Svelte browser bundle —
compiled separately by scripts/build-dashboard-ui.mjs ... covered by
tsconfig.dashboard.json for editor + CI type-checking." None of that is true:
- src/dashboard/ is server-rendered HTML (htmx + Alpine, both inlined by
scripts/vendor-ui.mjs) — not Svelte, and there is no esbuild-svelte step
- scripts/build-dashboard-ui.mjs and tsconfig.dashboard.json do not exist
- CI (.github/workflows/ci.yml) runs only the root typecheck/test/build
The exclude itself was a no-op: src/dashboard.ts (which IS in the program)
imports ./dashboard/fragments.js and ./dashboard/types.js, so tsc already
type-checks and emits dist/dashboard/* transitively. Verified: removing the
exclude leaves typecheck green and dist/dashboard/* emits identically with or
without it. Dropped both the false comment and the dead exclude entry.
The Responses API allows a message item's content to be a string OR an array of
input_text parts. transformOpenAIResponses only read the string form
(`typeof content === 'string' ? content : ''`), so a developer/system item sent
as [{ type: 'input_text', text: ... }] was neither imaged nor stubbed — its
verbose text rode uncompressed as native input, and the pointer replacement
(also string-only) left it in place.
- collect static context via the existing responsesContentText() helper, which
reads both shapes
- pointer replacement now handles arrays too, preserving the array shape
([{ type: 'input_text', text: RESPONSES_POINTER }]) so a parts-form request is
not reshaped into a string
- test: a developer item with array content is imaged (staticChars counts it)
and its outgoing content is a single pointer part, no original text
The original bug lived in collectSource's --git untracked branch, not in the
readExportTextFile helper the unit tests cover. Add an E2E test that runs the
actual CLI (tsx src/node.ts export --git --include '*.ts') against a throwaway
git repo with a tracked baseline plus untracked keep.ts, a 5000-char skip.md, an
oversized huge.ts, and a binary bin.ts. Asserts oversized + binary untracked
files are skipped (with warnings) and that only keep.ts's content reaches the
source (sourceChars < 500 — skip.md would have added thousands).
In --git mode, untracked files (git ls-files --others) were read straight to
memory with only a binary check — unlike directory and single-file modes, which
also apply --include/--exclude globs and the 1 MiB MAX_FILE_BYTES cap. So
`pxpipe export --git --include '*.ts'` still slurped every untracked file
regardless of the filter, and an untracked multi-MB file was read whole (a
resource-safety hazard).
- extract the shared gate into src/export-collect.ts: readExportTextFile()
applies include/exclude → size → binary, in that order, and is import-safe
(src/core/export.ts is contractually fs-free, so it can't live there)
- route all three collection paths (walkDir, single-file target, --git
untracked) through it; untracked skips now warn like explicit targets do
- tests/export-collect.test.ts covers include, exclude, oversized, the size
boundary, binary, and inaccessible
Verified end-to-end: `export --git --include '*.ts'` in a temp repo excludes an
untracked .md, skips an oversized .ts (with a warning), and keeps the .ts.
printVersion() logged `process.env.npm_package_version ?? '0.2.0'`. npm only
populates npm_package_version inside its own run-script env, so an end user
running `npx pxpipe-proxy --version` or a globally-installed `pxpipe` got the
stale hardcoded fallback (0.2.0) instead of the real version — the code comment
even claimed an esbuild.define that was never wired up.
- scripts/build.mjs reads package.json and inlines the version via esbuild
`define: { __PXPIPE_VERSION__ }`, then smoke-checks the built binary
(`node dist/node.js --version` must equal package.json) and fails the build
on mismatch, so a broken injection can never ship.
- src/node.ts reads the injected constant behind a `typeof` guard (safe under
tsx, where it is undefined), falling back to npm_package_version then
'unknown' — never a stale release number.
Verified: node dist/node.js --version now prints 0.8.0.
The 'Library use' snippet imported `renderTextToPngs` from "pxpipe", but the
published package is "pxpipe-proxy" and the root barrel (src/core/index.ts)
exports `renderTextToImages`, not `renderTextToPngs`. Both the import
specifier and the function name were unrunnable as written.
- import from "pxpipe-proxy" (matches package.json "name")
- use renderTextToImages, which returns { pages, droppedChars, pixels }
- show pages[i].png (Uint8Array) instead of the stale RenderedImage[] comment
If ANTHROPIC_API_KEY / OPENAI_API_KEY is configured on a deployed
Worker, the URL becomes an open key-spender: anyone who discovers it
gets requests signed with the deployment's key. Now:
- keys set + no PXPIPE_WORKER_SECRET -> 503, refuses to proxy at all
- keys set + secret -> callers must send x-pxpipe-secret (compared via
SHA-256 digests to avoid a prefix-timing signal); 401 otherwise
- no keys set -> unchanged (callers bring their own credentials)
The secret is stripped before forwarding upstream.
The dashboard HTML is served from the same origin as the JSON API, so
same-origin fetches never consult CORS. The wildcard header only served
cross-origin readers - i.e. any webpage open in the operator's browser
could read captured context out of 127.0.0.1. Browser default (opaque
cross-origin responses) is exactly what we want here.
The Node proxy called server.listen(port) with no host, so it bound all
interfaces (0.0.0.0/::) while the startup log printed http://127.0.0.1.
The dashboard is unauthenticated and serves captured request context
(/api/image-source returns the source text rendered into images) plus a
compression kill switch, with access-control-allow-origin: *. On a shared
network that context was readable by anyone who could reach the port.
Bind to 127.0.0.1 by default; add a HOST env to explicitly opt into
all-interfaces exposure (container/host-access use cases), with a startup
warning when bound off-loopback. IPv6 handled: ::1/localhost count as
loopback (no warning) and the startup URL is bracket-formatted. The one
README client example that used localhost is pinned to 127.0.0.1 to match
the IPv4 loopback default on IPv6-first hosts.
Verified: default refuses off-host connections; HOST=0.0.0.0 restores the
previous behavior + warns; HOST=::1 binds IPv6 loopback with a valid
bracketed URL and no warning. typecheck + build pass.
- staleFreshnessHints() rewrites the upstream "(file state is current in
your context - no need to Read it back)" hint into a stale-state warning
at history-serialization time, covering both collapsed same-session
turns and slabs inherited by continuation sessions
- Edit/Write/NotebookEdit stubs gain a live-text same-session Read
precondition (READ_FIRST_TOOLS), riding un-imaged in the tools array
- telemetry: churning_static_tags canary - FNV-1a fingerprint per
session/tag flags slab tags whose content churns and busts the image
cache (TAG_OBSERVATIONS_MAX-capped)
- 5 regression tests in tests/history.test.ts; baseline to beat: 97
read-gate errors across 18 sessions
Opening-turn task prompts were reduced to a 300-char compactPreview
tombstone by demoteProtectedHeadText, losing questions and trailing
output-format instructions that no later turn restated. The tombstone
keeps its cheap cache-stable preview, but latestCollapsedUserPointer
now scans INCLUDING the protected head: when the opening turn is the
only typed user text in the collapsed range (single-task sessions),
the pointer carries it verbatim via verbatimTaskText — up to 4000
chars, head 2600 / tail 1400 elision beyond that.
- README: Fable clip as headline demo (Drive link + committed thumb),
verified numbers from the recording: plain $42.21 / 96% context vs
pxpipe $4.51; honest caveat on single-reply format compliance
- ATTEMPTS.md: attempt 1 (invalid run, model-switch pitfall) and
attempt 2 (legibility PASS on fable, format miss documented)
- a.sh/b.sh: prompt fix — numbering starts at filler-000
Volatile env/context text (git status, cwd, date) rode in req.system —
BEFORE the slab anchor in Anthropic's prefix order (tools → system →
messages) — so any git-state change cold-restarted the entire anchored
prefix. Telemetry attribution (events.jsonl 2026-06-26..07-02): 48.8%
of cold-create waste, ~2.6k tokens/session vs ~200 saved by imaging it.
Now appended as a trailing text block on the LAST user message — the
per-turn live tail that re-caches incrementally anyway — placed AFTER
history collapse (never baked into frozen chunks) and after tool_result
compression. Session-stable billingLine/sysRemainder stay in system;
fallback keeps env in system when no user message exists to carry it.
New telemetry field: envRelocatedChars.
The Chrome extension attaches over its own channel (--strict-mcp-config
does not cover it) and injects tools mid-session, mutating the tool
array and busting the prompt-cache prefix from that point on. This
landed asymmetrically between arms and inflated b.sh's cache-write
cost. With --no-chrome the tool surface is frozen from request 1 in
both arms; a fresh A/B pair confirmed: $1.26 control vs $0.78 pxpipe,
cache writes 46.8k vs 22.5k, no write fail, no safeguard trip.
Fix#1 (cross-session cache bust): strip the volatile "# Environment"
markdown section (working dir, git status, platform, model ID) out of
the system text BEFORE the static/dynamic split. It carries no XML
wrapper so splitStaticDynamic couldn't catch it, and its git-status
lines change across sessions - baking them into the slab PNG busted
the cross-session cache (system_sha8 717f1fce -> 5efaa4bb for a
one-file edit). Parallel to stripBillingLine: the section re-enters
the system tail as plain text.
Schema handling simplified, knob removed: the Anthropic text-reference
path now renders prose-only tool docs (schemas stay untouched in
tools[] - duplicating them in the reference was pure bloat), while the
GPT imaged path always carries the compact schema (there it IS the
compression). Deletes the {type:'object'} stub + SCHEMA_STRUCTURAL_KEYS
machinery, stripSchemaDescriptions import, and the COMPRESS_SCHEMAS env.
Tests: e2e regressions for anchor relocation onto byte-frozen carry-over
history images and first-collapse anchor staying on the slab; history
relocation fixture grown past the collapseChunk=50 snap (56 messages ->
cutoff 50, frozen windows 11/21/31/41, collapsedTurns 49 - the old
35-message fixture floored to one window and never froze a chunk).
tsc clean; 615/615 tests pass.
The text-based Tool Reference triggered Anthropic's model-cloning
safeguard (stop_reason: refusal -> silent Opus fallback) because the
per-tool stub said docs lived in "the system prompt" — the exact
phrasing the 169521c retrip identified as the trigger for the imaged
slab, now reintroduced in text form.
- Stub now reads: 'ⓘ Full docs: see "## Tool: <name>" in the Tool
Reference section.' — no "system prompt"/"authoritative" framing.
- Tool Reference header is provenance-framed as first-party ("pxpipe
(this user's local proxy) moved the full tool documentation...").
- tracker.ts now maps tool_docs_chars into events.jsonl so refusals
are attributable to the text-reference path.
- Regression test: stub + reference header must never match
/system prompt|authoritative/i and must carry provenance framing
(scoped to the header; quoted third-party tool docs below it may
legitimately contain the phrase).
Verified: 614/614 tests, tsc --noEmit clean, and a live cold-start
through the rebuilt proxy on claude-fable-5 with the text reference
active — no refusal event, no fallback.
The in-image banner announced 'SYSTEM PROMPT + TOOL DOCS ... treat as
authoritative system instructions' inside a user turn. Anthropic's
model-cloning classifier read that as a replayed/extracted prompt and
refused (apiRefusalCategory: reasoning_extraction), forcing Claude Code
to fall back to claude-opus-4-8 — outside compress scope, so the whole
session ran passthrough with zero savings.
Reworded to first-party provenance framing (pxpipe rendered this
session's own configuration; no 'system prompt'/'authoritative').
Two headless probes through :47824 now serve on claude-fable-5[1m]
with the imaged slab (img:4, ~94k chars) and zero refusal events;
identical shape was refused pre-fix (req_011CccRnutLzRJNVZ7bRKG8Q).
Anthropic's safety classifier OCRs rendered images. A *legible* image of the
system prompt + tool docs (bash / permissions / credential / file-write
vocabulary) trips its prompt-injection heuristic and forces a model downgrade
to Opus, which costs far more than imaging the (already prompt-cached) slab
ever saves. Empirically the caption wording is NOT the trigger — the OCR'd
content is — so no caption rewrite fixes it; the slab must stay text.
Gate at the slab profitability check with PXPIPE_TEXT_SLAB=1. Default is
unchanged (legacy imaged slab) so the cache-align / cache-stability suites
stay valid. Both demos now launch the pxpipe proxy with the toggle set.
Verified: identical ~300KB-slab request images 8 PNGs by default, 0 with the
toggle (reason=slab_text). Full suite: 614 passed.
Both demos' a.sh/b.sh now request claude-fable-5[1m] for the default `fable`
arm, consistent with the opus/sonnet arms which already use [1m].
setup.sh PXPIPE_MODELS stays the BASE claude-fable-5 on purpose: the proxy
strips [variant] tags from the incoming model before matching against the
allow-list verbatim (src/core/applicability.ts), so base claude-fable-5
already covers claude-fable-5[1m]. Writing [1m] into the allow-list would
make the stripped incoming base no longer match -> pxpipe silently stops
compressing. Added a comment in each setup.sh pinning that.
Measured 2026-07-01 (count_tokens sweep, claude-sonnet-4-5; see
docs/LEGIBILITY-AUDIT-2026-07-01.md): the API downscales every image to fit
BOTH long-edge ≤1568 AND ~1.15 MP (≈1,143,750 px), then bills ≈px/750
(~1525 tok/img cap). The old 1932×1932 page hit the cap but was resampled
0.555× before the vision encoder, so 5×8 glyphs arrived at ~2.8×4.4 px — the
root cause of the legibility misses this session.
New page shape 1568×728 = 1,141,504 px fits both bounds → glyphs reach the
encoder WYSIWYG (and still ≤2000 px/side for >20-image requests).
render.ts geometry:
MAX_HEIGHT_PX 1932 -> 728
MAX_WIDTH_PX 1932 -> 1568
DEFAULT_COLS 313 -> 312 (1568 px exactly; 313=1573 would 0.997× blur)
DENSE_CONTENT_COLS 384 -> 312
READABLE/DENSE chars 50000/92160 -> 28080 (312×90, tracks real pagination)
Decouple the GPT path: OpenAI resizes differently (2048 bbox, 768 short side),
so a 768-wide strip up to 2048 tall survives un-resampled. Add
GPT_MAX_HEIGHT_PX = 1932 (gpt-model-profiles.ts) where the built-in cost
numbers were calibrated; openai-history.ts now uses it instead of the
now-Anthropic-specific MAX_HEIGHT_PX. Doc-only touch-ups in library.ts/export.ts.
tests/render + tests/paging updated to the new geometry (171 pass).
Holds the glyph cell at prod 5x8 and varies ONLY render style
(prod/onebit/color/grid/cgrid) at identical pixel dims => identical
image-token cost. Answers what the size sweep could not: at fixed cost,
does any style beat the ~10% exact-read baseline? Same content, grader,
and claude-CLI read method as eval/glyph-matrix/sweep.
The function-form transform intentionally returns {} on the active
path; the gate runs on static DEFAULTS (charsPerToken=4, priorWarm*=0)
and there is no live-alpha feedback loop from the dashboard. Record the
telemetry that justified this (2026-06: 897 sessions / 21,347 measured
rows, 5 mode flips ever, losses 0.8% of wins, all cache-create
amortization) so nobody "fixes" it without re-running that
reconciliation.
Negative Saved/lost cells get a small "create" pill (with explanatory
tooltip) when repricing the newly written prefix at the read rate turns
the loss positive: saved + cache_create x (1.25 - 0.1) > 0. Those rows
are the one-time cache-write premium being amortized, not gate
failures. Rates/token counts interpolate from CACHE_CREATE_RATE /
CACHE_READ_RATE so the copy cannot drift from the math.
Verified against ~/.pxpipe/events.jsonl replay: 3/3 negative cells in
the live 50-row window marked; warm-turn losses (cc=0) correctly bare.
- extractFactSheetEntries() returns {token, count}; the kept-token set
and order are byte-identical to the old extractFactSheetTokens(),
which now delegates. Counts render as a ×N suffix, so tally questions
over imaged content are answerable from the text sheet instead of by
counting 5x8 px glyph rows.
- Same-token spans matched by overlapping patterns dedupe by offset
(token + match index) so nothing double-counts.
- New tier-0 shape: uppercase hyphenated codes with a digit
(PROJ-1482, CVE-2024-30078); digit lookahead is bounded.
- export.ts consumes entries across all pages; 14 new/extended tests.
a.sh/b.sh -> claude-sonnet-5[1m] (was the stale claude-sonnet-4-6),
matching the opus alias's [1m] pattern. setup.sh -> claude-sonnet-5
(bare id, feeds PXPIPE_MODELS). claude-sonnet-5 was already in the dashboard
model catalog (fragments.ts); old 4.6 still reachable via the verbatim-id
fallback. No src/ changes.
A fresh same-session prior within the TTL is no longer sufficient to price
the text counterfactual warm: deriveBaselineWarmth now also requires the
static-prefix hash (system_sha8) to match. When opencode rotates the system
prompt / tool docs mid-session, the cacheable prefix changes, so a text-only
client would hit a new provider cache key too — pricing it warm against a
cold actual fabricated a huge phantom "loss" (the dashboard's 800%-worse
report). cr>0 still rescues a genuine warm read with no in-memory prior.
Wired through update(), replay(), and aggregateSessions via system_sha8.
Also surface losses honestly in the recent table (Saved/lost: negative
deltas in red instead of hidden as "—") and clarify headings as
billing-equivalent input tokens. Docs updated to match.
Danil Silantyev