Commit Graph

295 Commits

Author SHA1 Message Date
teamchong aa39fcea31 docs(demo): pin demo URLs to 127.0.0.1 to match the IPv4 loopback bind 2026-07-03 21:52:29 -04:00
Sutha Kamal 75263d64c3 fix(node): bind dashboard to loopback by default (HOST to opt into all-interfaces)
The Node proxy called server.listen(port) with no host, so it bound all
interfaces (0.0.0.0/::) while the startup log printed http://127.0.0.1.
The dashboard is unauthenticated and serves captured request context
(/api/image-source returns the source text rendered into images) plus a
compression kill switch, with access-control-allow-origin: *. On a shared
network that context was readable by anyone who could reach the port.

Bind to 127.0.0.1 by default; add a HOST env to explicitly opt into
all-interfaces exposure (container/host-access use cases), with a startup
warning when bound off-loopback. IPv6 handled: ::1/localhost count as
loopback (no warning) and the startup URL is bracket-formatted. The one
README client example that used localhost is pinned to 127.0.0.1 to match
the IPv4 loopback default on IPv6-first hosts.

Verified: default refuses off-host connections; HOST=0.0.0.0 restores the
previous behavior + warns; HOST=::1 binds IPv6 loopback with a valid
bracketed URL and no warning. typecheck + build pass.
2026-07-03 21:46:55 -04:00
teamchong ea7d526cad docs(readme): cut to 217 lines — dedupe caveats, move benchmark prose to eval/ links
Launch feedback: too long. Structural trims only; FAQ answers, demo
captions, and all measured numbers kept verbatim.
2026-07-03 19:30:33 -04:00
teamchong 3294c0d15c fix(history): stop teaching the model to skip Reads
- staleFreshnessHints() rewrites the upstream "(file state is current in
  your context - no need to Read it back)" hint into a stale-state warning
  at history-serialization time, covering both collapsed same-session
  turns and slabs inherited by continuation sessions
- Edit/Write/NotebookEdit stubs gain a live-text same-session Read
  precondition (READ_FIRST_TOOLS), riding un-imaged in the tools array
- telemetry: churning_static_tags canary - FNV-1a fingerprint per
  session/tag flags slab tags whose content churns and busts the image
  cache (TAG_OBSERVATIONS_MAX-capped)
- 5 regression tests in tests/history.test.ts; baseline to beat: 97
  read-gate errors across 18 sessions
2026-07-03 19:00:39 -04:00
Steven 9fa16eef54 Fix image link for pxpipe vs plain Claude demo
Updated image link for Opus 4.8 demo in README.
2026-07-03 11:35:26 -04:00
Steven 79c5e56862 Revise demo details for Fable 5 and Opus 4.8
Updated demo sections for Fable 5 and Opus 4.8, including new comparisons and performance metrics.
2026-07-03 11:25:23 -04:00
teamchong 2070c9697e release: v0.7.2 v0.7.2 2026-07-03 09:02:00 -04:00
teamchong c21520bd14 fix(history): carry opening-turn task text verbatim past demotion (#7)
Opening-turn task prompts were reduced to a 300-char compactPreview
tombstone by demoteProtectedHeadText, losing questions and trailing
output-format instructions that no later turn restated. The tombstone
keeps its cheap cache-stable preview, but latestCollapsedUserPointer
now scans INCLUDING the protected head: when the opening turn is the
only typed user text in the collapsed range (single-task sessions),
the pointer carries it verbatim via verbatimTaskText — up to 4000
chars, head 2600 / tail 1400 elision beyond that.
2026-07-03 08:23:41 -04:00
teamchong 0dce007d4c release: v0.7.1 v0.7.1 2026-07-03 01:50:27 -04:00
teamchong de0483cf51 fix(transform): wrap relocated env text in <system-reminder> for correct attribution 2026-07-03 01:50:08 -04:00
teamchong bb7aaa9fec release: v0.7.0 v0.7.0 2026-07-03 00:40:59 -04:00
teamchong ed8be0bfea docs: changelog for 0.7.0 2026-07-03 00:40:59 -04:00
teamchong b7de47eb3f docs: add Fable 5 A/B demo (same answers, ~9x cheaper) + attempt log
- README: Fable clip as headline demo (Drive link + committed thumb),
  verified numbers from the recording: plain $42.21 / 96% context vs
  pxpipe $4.51; honest caveat on single-reply format compliance
- ATTEMPTS.md: attempt 1 (invalid run, model-switch pitfall) and
  attempt 2 (legibility PASS on fable, format miss documented)
- a.sh/b.sh: prompt fix — numbering starts at filler-000
2026-07-03 00:37:06 -04:00
teamchong ece33030e0 Add headless bench: multi-turn claude -p driver + events.jsonl scorer
bench/run.sh: builds, boots an isolated proxy (auto free port, own-log
readiness check so a stale listener can't absorb the run), seeds a git
workspace from demo/cost-ab/template, churns volatile env state between
turns, then scores the run's events.jsonl.

bench/score.mjs: per-session + overall cost_units (1.25x create / 0.1x
read), prefix flips, cold restarts, flip waste, stop_reason histogram,
safety flags; writes score.json; two args = A/B diff.
2026-07-02 23:27:34 -04:00
teamchong 06a8b70c01 fix(transform): relocate volatile env text behind ALL cache breakpoints
Volatile env/context text (git status, cwd, date) rode in req.system —
BEFORE the slab anchor in Anthropic's prefix order (tools → system →
messages) — so any git-state change cold-restarted the entire anchored
prefix. Telemetry attribution (events.jsonl 2026-06-26..07-02): 48.8%
of cold-create waste, ~2.6k tokens/session vs ~200 saved by imaging it.

Now appended as a trailing text block on the LAST user message — the
per-turn live tail that re-caches incrementally anyway — placed AFTER
history collapse (never baked into frozen chunks) and after tool_result
compression. Session-stable billingLine/sysRemainder stay in system;
fallback keeps env in system when no user message exists to carry it.
New telemetry field: envRelocatedChars.
2026-07-02 23:17:13 -04:00
teamchong e49bb11878 feat(telemetry): log stop_reason + safety-flag per request 2026-07-02 23:07:37 -04:00
teamchong 42a3b363d5 fix(demo): add --no-chrome to both cost-ab arms
The Chrome extension attaches over its own channel (--strict-mcp-config
does not cover it) and injects tools mid-session, mutating the tool
array and busting the prompt-cache prefix from that point on. This
landed asymmetrically between arms and inflated b.sh's cache-write
cost. With --no-chrome the tool surface is frozen from request 1 in
both arms; a fresh A/B pair confirmed: $1.26 control vs $0.78 pxpipe,
cache writes 46.8k vs 22.5k, no write fail, no safeguard trip.
2026-07-02 22:24:13 -04:00
teamchong beacd67b4a fix(transform): freeze imaged slab at first render — keep volatile content (skill listings, cwd caches) out of the imaged prefix so turn-2 system sha matches turn-1 2026-07-02 21:11:18 -04:00
teamchong 0ad3c68664 fix(transform): cross-session slab stability + drop compressSchemas knob
Fix #1 (cross-session cache bust): strip the volatile "# Environment"
markdown section (working dir, git status, platform, model ID) out of
the system text BEFORE the static/dynamic split. It carries no XML
wrapper so splitStaticDynamic couldn't catch it, and its git-status
lines change across sessions - baking them into the slab PNG busted
the cross-session cache (system_sha8 717f1fce -> 5efaa4bb for a
one-file edit). Parallel to stripBillingLine: the section re-enters
the system tail as plain text.

Schema handling simplified, knob removed: the Anthropic text-reference
path now renders prose-only tool docs (schemas stay untouched in
tools[] - duplicating them in the reference was pure bloat), while the
GPT imaged path always carries the compact schema (there it IS the
compression). Deletes the {type:'object'} stub + SCHEMA_STRUCTURAL_KEYS
machinery, stripSchemaDescriptions import, and the COMPRESS_SCHEMAS env.

Tests: e2e regressions for anchor relocation onto byte-frozen carry-over
history images and first-collapse anchor staying on the slab; history
relocation fixture grown past the collapseChunk=50 snap (56 messages ->
cutoff 50, frozen windows 11/21/31/41, collapsedTurns 49 - the old
35-message fixture floored to one window and never froze a chunk).

tsc clean; 615/615 tests pass.
2026-07-02 17:36:18 -04:00
teamchong baa1e1ee01 fix(transform): reword tool-docs stub/header to avoid refusal classifier
The text-based Tool Reference triggered Anthropic's model-cloning
safeguard (stop_reason: refusal -> silent Opus fallback) because the
per-tool stub said docs lived in "the system prompt" — the exact
phrasing the 169521c retrip identified as the trigger for the imaged
slab, now reintroduced in text form.

- Stub now reads: 'ⓘ Full docs: see "## Tool: <name>" in the Tool
  Reference section.' — no "system prompt"/"authoritative" framing.
- Tool Reference header is provenance-framed as first-party ("pxpipe
  (this user's local proxy) moved the full tool documentation...").
- tracker.ts now maps tool_docs_chars into events.jsonl so refusals
  are attributable to the text-reference path.
- Regression test: stub + reference header must never match
  /system prompt|authoritative/i and must carry provenance framing
  (scoped to the header; quoted third-party tool docs below it may
  legitimately contain the phrase).

Verified: 614/614 tests, tsc --noEmit clean, and a live cold-start
through the rebuilt proxy on claude-fable-5 with the text reference
active — no refusal event, no fallback.
2026-07-02 13:17:39 -04:00
teamchong 169521c9f8 fix(transform): defuse reasoning_extraction refusal — provenance-framed slab banner
The in-image banner announced 'SYSTEM PROMPT + TOOL DOCS ... treat as
authoritative system instructions' inside a user turn. Anthropic's
model-cloning classifier read that as a replayed/extracted prompt and
refused (apiRefusalCategory: reasoning_extraction), forcing Claude Code
to fall back to claude-opus-4-8 — outside compress scope, so the whole
session ran passthrough with zero savings.

Reworded to first-party provenance framing (pxpipe rendered this
session's own configuration; no 'system prompt'/'authoritative').
Two headless probes through :47824 now serve on claude-fable-5[1m]
with the imaged slab (img:4, ~94k chars) and zero refusal events;
identical shape was refused pre-fix (req_011CccRnutLzRJNVZ7bRKG8Q).
2026-07-01 22:25:08 -04:00
teamchong 7f2e40bad6 Revert "feat(transform): PXPIPE_TEXT_SLAB=1 keeps the system slab as text (classifier-safe)"
This reverts commit c04e8f80e8.
2026-07-01 21:50:39 -04:00
teamchong c04e8f80e8 feat(transform): PXPIPE_TEXT_SLAB=1 keeps the system slab as text (classifier-safe)
Anthropic's safety classifier OCRs rendered images. A *legible* image of the
system prompt + tool docs (bash / permissions / credential / file-write
vocabulary) trips its prompt-injection heuristic and forces a model downgrade
to Opus, which costs far more than imaging the (already prompt-cached) slab
ever saves. Empirically the caption wording is NOT the trigger — the OCR'd
content is — so no caption rewrite fixes it; the slab must stay text.

Gate at the slab profitability check with PXPIPE_TEXT_SLAB=1. Default is
unchanged (legacy imaged slab) so the cache-align / cache-stability suites
stay valid. Both demos now launch the pxpipe proxy with the toggle set.

Verified: identical ~300KB-slab request images 8 PNGs by default, 0 with the
toggle (reason=slab_text). Full suite: 614 passed.
2026-07-01 19:46:29 -04:00
teamchong ed968a7495 Reapply "fix(render): fit Anthropic page to 1568×728 (~1.15 MP) for WYSIWYG glyphs"
This reverts commit 6eaa8bfcd0.
2026-07-01 19:18:20 -04:00
teamchong 6eaa8bfcd0 Revert "fix(render): fit Anthropic page to 1568×728 (~1.15 MP) for WYSIWYG glyphs"
This reverts commit 838db8778e.
2026-07-01 18:50:25 -04:00
teamchong 9cade86868 debug(node): add PXPIPE_DUMP_DIR to persist rendered PNGs for demo inspection 2026-07-01 18:50:25 -04:00
teamchong ad947335e5 demo: run fable arm on claude-fable-5[1m] (1M ctx), matching opus/sonnet
Both demos' a.sh/b.sh now request claude-fable-5[1m] for the default `fable`
arm, consistent with the opus/sonnet arms which already use [1m].

setup.sh PXPIPE_MODELS stays the BASE claude-fable-5 on purpose: the proxy
strips [variant] tags from the incoming model before matching against the
allow-list verbatim (src/core/applicability.ts), so base claude-fable-5
already covers claude-fable-5[1m]. Writing [1m] into the allow-list would
make the stripped incoming base no longer match -> pxpipe silently stops
compressing. Added a comment in each setup.sh pinning that.
2026-07-01 18:06:53 -04:00
teamchong 838db8778e fix(render): fit Anthropic page to 1568×728 (~1.15 MP) for WYSIWYG glyphs
Measured 2026-07-01 (count_tokens sweep, claude-sonnet-4-5; see
docs/LEGIBILITY-AUDIT-2026-07-01.md): the API downscales every image to fit
BOTH long-edge ≤1568 AND ~1.15 MP (≈1,143,750 px), then bills ≈px/750
(~1525 tok/img cap). The old 1932×1932 page hit the cap but was resampled
0.555× before the vision encoder, so 5×8 glyphs arrived at ~2.8×4.4 px — the
root cause of the legibility misses this session.

New page shape 1568×728 = 1,141,504 px fits both bounds → glyphs reach the
encoder WYSIWYG (and still ≤2000 px/side for >20-image requests).

render.ts geometry:
  MAX_HEIGHT_PX          1932  -> 728
  MAX_WIDTH_PX           1932  -> 1568
  DEFAULT_COLS            313  -> 312  (1568 px exactly; 313=1573 would 0.997× blur)
  DENSE_CONTENT_COLS     384  -> 312
  READABLE/DENSE chars  50000/92160 -> 28080  (312×90, tracks real pagination)

Decouple the GPT path: OpenAI resizes differently (2048 bbox, 768 short side),
so a 768-wide strip up to 2048 tall survives un-resampled. Add
GPT_MAX_HEIGHT_PX = 1932 (gpt-model-profiles.ts) where the built-in cost
numbers were calibrated; openai-history.ts now uses it instead of the
now-Anthropic-specific MAX_HEIGHT_PX. Doc-only touch-ups in library.ts/export.ts.

tests/render + tests/paging updated to the new geometry (171 pass).
2026-07-01 17:54:51 -04:00
teamchong d7999279a5 eval: add constant-cost render-style A/B harness (paused task #7 arms)
Holds the glyph cell at prod 5x8 and varies ONLY render style
(prod/onebit/color/grid/cgrid) at identical pixel dims => identical
image-token cost. Answers what the size sweep could not: at fixed cost,
does any style beat the ~10% exact-read baseline? Same content, grader,
and claude-CLI read method as eval/glyph-matrix/sweep.
2026-07-01 16:41:24 -04:00
teamchong b453adf0c1 docs(node): pin why the transform hook is only a kill switch
The function-form transform intentionally returns {} on the active
path; the gate runs on static DEFAULTS (charsPerToken=4, priorWarm*=0)
and there is no live-alpha feedback loop from the dashboard. Record the
telemetry that justified this (2026-06: 897 sessions / 21,347 measured
rows, 5 mode flips ever, losses 0.8% of wins, all cache-create
amortization) so nobody "fixes" it without re-running that
reconciliation.
2026-07-01 16:41:12 -04:00
teamchong edf7a05f85 feat(dashboard): tag one-time cache-create losses in recent table
Negative Saved/lost cells get a small "create" pill (with explanatory
tooltip) when repricing the newly written prefix at the read rate turns
the loss positive: saved + cache_create x (1.25 - 0.1) > 0. Those rows
are the one-time cache-write premium being amortized, not gate
failures. Rates/token counts interpolate from CACHE_CREATE_RATE /
CACHE_READ_RATE so the copy cannot drift from the math.

Verified against ~/.pxpipe/events.jsonl replay: 3/3 negative cells in
the live 50-row window marked; warm-turn losses (cc=0) correctly bare.
2026-07-01 16:41:12 -04:00
teamchong a346ec7ac5 feat(factsheet): carry occurrence counts, match ticket-style codes
- extractFactSheetEntries() returns {token, count}; the kept-token set
  and order are byte-identical to the old extractFactSheetTokens(),
  which now delegates. Counts render as a ×N suffix, so tally questions
  over imaged content are answerable from the text sheet instead of by
  counting 5x8 px glyph rows.
- Same-token spans matched by overlapping patterns dedupe by offset
  (token + match index) so nothing double-counts.
- New tier-0 shape: uppercase hyphenated codes with a digit
  (PROJ-1482, CVE-2024-30078); digit lookahead is bounded.
- export.ts consumes entries across all pages; 14 new/extended tests.
2026-07-01 16:41:00 -04:00
teamchong d1704241d6 demo: add claude-sonnet-5 support alongside opus
a.sh/b.sh  -> claude-sonnet-5[1m] (was the stale claude-sonnet-4-6),
matching the opus alias's [1m] pattern. setup.sh  -> claude-sonnet-5
(bare id, feeds PXPIPE_MODELS). claude-sonnet-5 was already in the dashboard
model catalog (fragments.ts); old 4.6 still reachable via the verbatim-id
fallback. No src/ changes.
2026-06-30 15:22:20 -04:00
teamchong f472566cc1 release: v0.6.10
Claude-Session: https://claude.ai/code/session_014gJXMzmULNmtuNhbhSdjVo
v0.6.10
2026-06-30 12:45:08 -04:00
Steven Chong bf49065dce fix(savings): use observed cache state for text baseline 2026-06-30 12:24:09 -04:00
Steven Chong ff8a05e30c fix(savings): gate text-counterfactual warmth on static-prefix hash
A fresh same-session prior within the TTL is no longer sufficient to price
the text counterfactual warm: deriveBaselineWarmth now also requires the
static-prefix hash (system_sha8) to match. When opencode rotates the system
prompt / tool docs mid-session, the cacheable prefix changes, so a text-only
client would hit a new provider cache key too — pricing it warm against a
cold actual fabricated a huge phantom "loss" (the dashboard's 800%-worse
report). cr>0 still rescues a genuine warm read with no in-memory prior.

Wired through update(), replay(), and aggregateSessions via system_sha8.

Also surface losses honestly in the recent table (Saved/lost: negative
deltas in red instead of hidden as "—") and clarify headings as
billing-equivalent input tokens. Docs updated to match.
2026-06-29 15:59:10 -04:00
teamchong ec43bf5370 release: v0.6.9 v0.6.9 2026-06-27 23:41:11 -04:00
teamchong 966f9a144e refactor(export): single-source REPORT_CHARS_PER_TOKEN in transform.ts
Drop the duplicate CHARS_PER_TOKEN=3.7 from export.ts and import REPORT_CHARS_PER_TOKEN from transform.ts so the reporting-estimate constant has one source of truth. Clarify the dashboard baseline-warmth invariant comment. No behavior change.
2026-06-27 23:41:11 -04:00
teamchong cde21b2e73 fix(savings): base text-counterfactual warmth on wall-clock, not image-cache fate
The savings accounting priced the text baseline as cold whenever pxpipe's own
image cache missed this turn (cr==0), fabricating a 1.25x create the text path
would never have paid and inflating reported savings ~2x. Decouple them: the
text prefix reads warm whenever a fresh same-session prior exists within the
300s TTL (wall-clock), unioned with an observed read (cr>0) for the post-restart
case. Centralised in deriveBaselineWarmth; used by all three call sites
(sessions, dashboard, fragments).

Empirical replay over ~/.pxpipe/events.jsonl (13,402 message rows): honest
headline 504.6M tokens saved vs the old cr-gated 1.05B; 1,409/12,305 fresh-prior
rows (11.4%) read warm despite a busted image cache. Dashboard now narrates the
busted-image case explicitly. Adds baseline/sessions/context-map regression
tests. Measurement only — no change to proxy behavior.

typecheck + 600 tests + build all green.
2026-06-27 21:26:55 -04:00
teamchong 76a371028a feat(export): replace in-band @pxpipe directive with explicit pxpipe export CLI
The magic-word @pxpipe directive fired unreliably and added per-request body
scanning to the hot path, so drop it entirely: removed the directive module +
its test and unwired it from transform.ts / proxy.ts / node.ts / index.ts.
transformRequest now leaves message bodies byte-identical — no per-request
directive parsing.

In its place, `pxpipe export <paths>` renders code/text to dense PNG pages via
the same SDK primitive the proxy uses, so exported pages are byte-identical to
what ships to the model. Width is locked to proxy density (--cols rejected as
an unknown option, guarded by tests) to prevent drift. Added --open to preview
the rendered pages plus a result footer (artifact summary + token-savings line).

Tests: export.test.ts 64 passed (incl. --open parse + default coverage); full
suite 589 passed / 29 files; typecheck clean.
2026-06-27 18:28:28 -04:00
teamchong ecea71d1a0 dashboard: order Sonnet newest-first (Sonnet 5 before 4.6)
Aligns Sonnet with the newest-first convention the rest of MODEL_CATALOG
already uses (Opus 4.8>4.7, GPT 5.6>5.5). Cosmetic chip order only.
2026-06-26 00:58:53 -04:00
teamchong 9f25a4db3b ci: move actions to Node 24 majors (checkout@v7, setup-node@v6, pnpm/action-setup@v6)
Clears the Node 20 runtime deprecation warning on the runner.
package.json already sets packageManager: pnpm@10.21.0, so action-setup@v6
resolves the pnpm version with no action input - same as the prior bare usage.
2026-06-26 00:24:58 -04:00
teamchong 873cbaf94f release: v0.6.8 v0.6.8 2026-06-26 00:17:10 -04:00
teamchong 6d1c46dd20 fix(cache): pin prefix-cache anchor to the byte-stable history image
Intermittent prefix-cache busts on long sessions (#11): the single cache
breakpoint relocated onto the LAST history image — the still-growing chunk
whose bytes change every window advance — so the slab+history prefix
re-created at 1.25x instead of being read.

- relocateAnchorToHistoryImage pins collapseHistory's carry-over ordinal
  (the last byte-stable chunk) instead of the last image.
- demoteProtectedHeadText keeps pxpipe's slab scaffolding (images +
  fact-sheet + the [End of rendered context.] boundary) verbatim and only
  quarantines the user's stale opening turn, so the relocator can still
  find the slab anchor (#14 demotion had clobbered the boundary it keys on).
- cachePrefixDigest joins parts with \x00 so block-boundary differences
  can't hash-collide into false prefix-stability.

520/520 green, tsc clean.
2026-06-26 00:17:10 -04:00
teamchong 6ca72260e8 chore: release 0.6.7 v0.6.7 2026-06-25 22:15:20 -04:00
teamchong 85625f6a7a feat(transform): cache_prefix_sha8 prefix-cache bust attribution probe (#11)
Read-only digest of the exact pinned prefix (tools+system+messages through
the imaged history/slab boundary, live tail excluded), computed after final
marker placement and emitted per-turn via tracker. Attributes a prompt-cache
bust as pxpipe-side (sha8 drifts turn-over-turn) vs upstream eviction (sha8
stable while cache_create spikes). Behavior-neutral: byte-identical requests,
518/518 green.
2026-06-25 22:14:37 -04:00
teamchong 7d15271587 chore: release 0.6.6 v0.6.6 2026-06-25 19:59:37 -04:00
teamchong e9cc56656a feat(dashboard): dark/light mode toggle
Top-right toggle flips between the warm-light theme and a new dark palette (same flame identity, inverted neutrals). Pure client-side: a <head> script sets the theme before first paint (no flash) from localStorage, defaulting to prefers-color-scheme; no new route or server round-trip. Themed via the existing :root CSS variables plus dark fix-ups for the few hard-coded light spots.
2026-06-25 19:58:54 -04:00
teamchong 2e8f89288b fix(factsheet): rank tokens by shape so short high-consequence ids survive
The 64-token budget was filled longest-first, so long low-risk doc-URLs evicted short zero-redundancy tokens (git SHAs, ports, CLI flags, CONST_IDS) — the strings a coding agent must reproduce verbatim. Allocate by priority tier (ids/flags/numbers > paths/versions > URLs, capped); preserve substring-collapse and byte-stable, cache-safe output. Adds an eviction regression test.
2026-06-25 19:58:54 -04:00
Steven Chong 9d121acc20 fix(node): backpressure + graceful client-abort handling in response streaming
writeWebResponse now respects write() backpressure via a drain/close race,
cancels the upstream body reader when the client hangs up, and treats socket
aborts (UND_ERR_SOCKET/ECONNRESET/EPIPE/AbortError, "terminated",
"other side closed") as clean disconnects instead of throwing. Releases the
reader lock and removes the close listener in finally.
2026-06-25 17:28:33 -04:00