mirror of
https://github.com/teamchong/pxpipe.git
synced 2026-07-22 02:02:51 +02:00
2b7b98f68550d93b33dbffb8e1af54463cf11ff8
46 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 teamchong
|
2b7b98f685 |
feat(render): add R3 reflow to recover line-end dead margin (~29% glyph fill → dense)
Pack text into a continuous sentinel-delimited stream (↵ = U+21B5) so
wrapLines fills every row to `cols` instead of leaving dead right-margin.
Adds reflow/dereflow, renderTextToPngsReflow{,MultiCol} variants, a full
test suite, and an A/B eval harness with L1 OCR + L2 session results.
|
||
|
teamchong
|
a19ce30a48 |
feat(core): quantize history collapse boundary onto a fixed grid (#28)
Snap the collapse cutoff to a `collapseChunk`-sized grid (default 50) so the rendered history image stays byte-identical for a full window of turns. That lets Anthropic's prompt cache `cache_read` the history prefix (0.1x) instead of re-billing `cache_create` (1.25x) every turn. The cutoff floors at `minCollapsePrefix` (clamped to the raw cutoff) so short conversations still collapse without the boundary drifting. Also log `history_image_sha8` per request -- a sha8 of the concatenated history-image base64. An unchanged hash across consecutive collapsed events is ground-truth proof the prompt cache is being hit; a hash that moves every turn signals the cache-key drift regression is back. |
||
|
Steven Chong
|
2ae75db58e | docs(render): clarify 5x8 atlas sizing | ||
|
Steven Chong
|
d87aecec95 | feat(render): use 5x8 code-font atlas | ||
|
Steven Chong
|
1a83d56379 |
feat(transform): priorWarmTokens cache-burn penalty in break-even gate
Production data (ocproxy 7d, Opus 4.7) shows pixelpipe HURTS short conversations and helps long ones: input.count <10 : +$0.17 (HURTS) over 14 rows input.count 10-19: +$0.04 (HURTS) over 22 rows input.count 20-39: -$0.56 (helps) over 29 rows input.count 40+ : -$3.21 (helps) over 237 rows Diagnosis: warm-cache share split. bucket actual_warm_share baseline_warm_share <10 60% 100% 10-19 84% 97% 20-39 88% 96% 40+ 92% 58% For short conversations the un-rewritten path is already nearly fully warm-cached. Rewriting the prefix bytes (slab compression / image substitution / history collapse) gives the new prefix a different cache key, so Anthropic charges cache_create (1.25×) on the new prefix's first turn — destroying the prior warm cache. The conversation often ends before the new cache amortizes. Fix: add a 'priorWarmTokens' option to TransformOptions. When >0, the break-even gate penalises the image side by burn = priorWarmTokens × (CACHE_CREATE_RATE − CACHE_READ_RATE) Per-turn gate eats the full burn; amortized gate spreads textLifetime across N turns so longer horizons re-accept once savings outweigh burn. Default 0 preserves byte-identical behavior for callers that don't plumb it through (cold-start safe). Tests: 3 new (NaN clamp, per-turn flip, horizon flip). 302 total pass. |
||
|
local
|
9ecd799092 |
feat(transform): lower SLAB/HISTORY_CHARS_PER_TOKEN 2.5 → 2.0 for Opus 4.7 tokenizer
N=391 Opus 4.7 measured rows (last 7d) show real chars-per-token = 1.91 (avg_outgoing_text_chars=231,925 / avg_real_input_tokens=115,893). The older 2.5 constant was calibrated on Opus 4.5/4.6 and over-estimated cpt by ~30%, under-counting text-token cost and causing the break-even gate to reject most history-collapse opportunities (283/391 = 72% of measured ok rows came back history_reason='not_profitable'). Resize one render.test.ts fixture (6060 → 4848 chars) so the 'low cpt unlocks a previously-rejected slab' assertion still exercises the intended before/after edge under the new default. |
||
|
Steven Chong
|
ed2746650f |
fix(dashboard): exclude unmeasured rows from "$ saved" rollup
When the cacheable-prefix count_tokens probe failed but the full-body probe succeeded, the dashboard fell through to `baselineCacheableTokens ?? 0`. That silently treated the unproxied counterfactual as cold-input-only — charging full $2.50/Mtok input rate on tokens that demonstrably would have been cache-discounted at 10× cheaper. Net effect: fabricated "$ saved" headline on every partial-probe row. The fix: 1. New `TransformInfo.baselineProbeStatus: 'ok' | 'partial' | 'failed'`, set by the proxy after awaiting both probes. 'ok' means either both probes resolved OR the original body had no cache_control markers (cacheable=0 is then exact, not estimated). 'partial' means the full-body probe resolved but the cacheable-prefix probe didn't. 2. `tracker.ts` persists it as `baseline_probe_status` on the JSONL. 3. `dashboard.ts` gates its savings math on `status === 'ok'` (with a back-compat path for legacy rows that pre-date the field). Partial / failed rows now contribute to the request count but not to baselineInputWeighted / actualInputWeighted / outputWeighted. ocproxy's host-side dashboard already records baseline_probe_status in its SQLite schema and is being updated in parallel to honor the same gating semantic. See ocproxy commit fixing dashboard/src/lib/snapshot.ts for the matching change. |
||
|
Steven Chong
|
a197f8729d |
feat(transform): horizon-aware history-collapse break-even gate
Adds isCompressionProfitableAmortized() and a new historyAmortizationHorizon option (default 1 = current per-turn behaviour). When the host opts in to horizon ≥ 2, the history-collapse gate evaluates expected lifetime cost over N future turns instead of the cold per-turn comparison: I × (CC + CR×(N-1)) < T × CR × N where CC = 1.25 (cache_create), CR = 0.10 (cache_read) At N=5 this accepts collapses with I/T < 0.30; at N=10 with I/T < 0.47. The worst-case-for-image / best-case-for-text framing is on purpose — we only collapse when the math wins under pessimistic warm-cache assumptions, so a single-turn session can never pay a tax. This is the 'try-then-decide-with-bounded-horizon' design from the README's new 'unsolved part: multi-turn amortization' section. Picked over JIT-style session state (rejected: introduces durable state), always-collapse (rejected: dishonest per-request), and cache-bust-driven (rejected: signal arrives one turn late). - New option threaded through DEFAULTS, TransformOptions, PixelpipeOptions - Both history call sites (early-exit helper + main success path) use the amortized gate when horizon > 1, fall back to the cold gate otherwise - 4 new unit tests covering fallback, accept-on-long-history, reject-on- tiny-text, and the subset-relationship between cold and amortized gates - 298 tests pass |
||
|
Steven Chong
|
b9ea0a73b8 |
feat(transform): run history collapse on below_min/not_profitable exits
Pixelpipe's static-slab gate returns early when the system slab is below minCompressChars or fails the slab break-even check. Both exits skipped the always-on Variant C history-image collapse, so production traffic with a tiny system prompt but a huge messages[] array (e.g. Codex with no project instructions / CLAUDE.md / skills) never got any compression even though the dynamic side was massive. Trace on a real-shaped request (system=29 chars, 56 message turns, ~275k outgoing text chars) before this change: applied=false, reason=below_min_chars, imageCount=0, collapsedTurns=undefined After: applied=true, reason=applied, imageCount=20, collapsedTurns=56, collapsedChars=256832, outgoingTextChars=19782 Implementation: extracted runHistoryCollapseAndFinalize() helper that runs collapseHistory + countOutgoingTextChars + JSON.stringify. The two early-exit paths now call the helper; if history collapses they flip info.compressed = true (library.ts then returns reason='applied'), otherwise they return the original body unchanged (preserving the existing below_min_chars / not_profitable diagnostics for the slab). Main success path unchanged — it already ran history collapse after slab compression. No new flags, default-on for Opus 4.7 per host gating. 294 tests pass. |
||
|
teamchong
|
c1d111a083 |
feat(transform): per-bucket char attribution telemetry (Task #18 Phase 1)
Adds the data collection half of the adaptive chars-per-token plan
without touching the gate. Every isCompressionProfitable call site
now credits its chars to a named bucket on TransformInfo, and those
counts flow through toTrackEvent into the persisted JSONL.
transform.ts:
- New BucketName union: 'static_slab' | 'reminder' |
'tool_result_{structured,log,prose}' | 'history'
- bumpBucket() helper lazily allocates info.bucketChars
- toolResultBucket() routes by classifyContent shape so the post-
compaction text we credit matches the shape the gate evaluated
- Bump sites: static slab (combinedRaw), reminder (reminderRaw),
tool_result single-string (innerRaw), tool_result multi-block
per TextBlock (innerTextRaw), history (collapsedChars)
- Surfaces historyTextChars separately so history's no-collapse
rejections still contribute a denominator
tracker.ts:
- TrackEvent gains bucket_chars?: Partial<Record<BucketName, number>>
and history_text_chars?: number
- toTrackEvent copies them when populated, drops empty bucket maps
so happy-path events stay small
tests/tracker.test.ts:
- Asserts snake_case field names and nested shape land correctly
- Asserts absent buckets and pass-through events stay clean
No gate logic changed. CHARS_PER_TOKEN / SLAB_CHARS_PER_TOKEN /
HISTORY_CHARS_PER_TOKEN still drive isCompressionProfitable. Phase 2
will fit per-bucket slopes from the captured events and wire a
bucket-aware override.
docs/ADAPTIVE_CPT_PLAN.md: full plan + empirical basis (slab-only
regression on 87 events recovered cpt≈1.50 vs the baked 2.5
constant — 40% on non-slab compressions ship today).
Verified: tsc --noEmit clean; 290/290 tests pass (+2 new);
build emits dist/node.js.
|
||
|
teamchong
|
440ca7f448 |
test: real-shape regression fixtures from production events.jsonl
Fragility #3 from end-of-session audit. The previous gate tests pinned isCompressionProfitable(slab, 100, 2, 2.5) === true against 'A'.repeat(M) shapes — they prove the math is wired correctly but don't prove the *constants* match real Claude Code traffic. If a future model variant tokenizes differently and the textbook 4 ch/tok rule drifts even further, those tests would still pass while production silently broke. Anonymized 6 production shapes from events.jsonl (2026-05-19..2026-05-20): - PRODUCTION_SLAB_161K (multi-col, 52 chars/row, 11 ings/141 lines/2 cols) - PRODUCTION_SLAB_135K_DENSE (neuline-heavy, 19 chars/row, 24 ings) - PRODUCTION_SLAB_169K_HEAVY (very dense, 16 chars/row — actually rejected) - BELOW_MIN_CHARS_TINY (142 chars, pre-filter) - BELOW_MIN_CHARS_BORDERLINE (1123 chars, pre-filter) - HISTORY_COLLAPSED_LONG_SESSION (collapsed path captured) Each fixture stores its real captured decision plus the shape parameters (origChars, numCols, approxCharsPerRow). synthesizeText() pads 'A'.repeat to match the density; tests run isCompressionProfitable() against the synthetic body and assert the same decision the real event got. \#\# Notable: 135K_DENSE divergence The real 135K neuline-heavy event was ACCEPTED (compressed=true) but the synthetic 'A'.repeat(19) reconstruction REJECTS even at cpt=2.5. Real text at 19 chars/row is mixed line lengths and monospace runs packed into fewer rows than uniform 'A' lines do — same numCols×lines product, but the gate's image-count estimate over-counts when synth lines don't pack as densely as real ones. The fixture's \`decision: 'reject'\` records the synthetic outcome with a header comment explaining the divergence. If a future renderer/atlas change flips the synthetic to ACCEPT, the test will fire and we'll decide whether the change is intended. \#\# Export changes SLAB_CHARS_PER_TOKEN and HISTORY_CHARS_PER_TOKEN exported from transform.ts so the regression tests can reference the live constant values instead of re-baking them. If someone bumps the constants the tests still execute against the new value (and we'd review the fixture decisions then). \#\# Tests 8 files / 280 tests pass, typecheck clean, build clean. 7 new it() blocks cover the 5 captured shapes + the synthetic divergence note. |
||
|
teamchong
|
ea32dd2ed5 |
fix(gate): discriminate raw opts to fix override-gate default-value collision
The override-gate previously used `o.charsPerToken !== CHARS_PER_TOKEN` to detect "host pinning". After CHARS_PER_TOKEN=4 was replaced with slab/history-specific 2.5 constants, a host that deliberately passes the literal `4` (the same as DEFAULTS) was silently swapped to 2.5 - the gate couldn't distinguish "host passed nothing" from "host passed 4". Fix: check the raw opts (\`opts.charsPerToken !== undefined\`) instead of comparing the merged value against a sentinel. Host-supplied `4` now flows through as `4`, even though it matches the default. Tests pin both directions: - slab path (render.test.ts): explicit cpt=4 keeps the 161k slab REJECTED at cpt=4 (would be ACCEPTED at the built-in 2.5) - history path (history.test.ts): explicit cpt=4 keeps a 14-turn closed conversation REJECTED at cpt=4 (would collapse at the built-in 2.5), with companion cpt=2.5 test proving the fixture actually straddles 275 tests pass, typecheck clean, build clean. |
||
|
teamchong
|
b563751cf6 |
fix(gate): history call site uses empirically-grounded cpt=2.5 (was 4)
Production telemetry (N=10 events with history_reason="not_profitable", 2026-05-18..05-20) shows body-level chars-per-token on rejected history events: median 1.09, max 1.10. The gate at cpt=4 was estimating text as 3.7× cheaper than reality on every history-collapse decision in the sampled traffic — same shape of bug as the slab path fixed in |
||
|
teamchong
|
c36ec3fd39 |
fix(gate): slab call site uses empirically-grounded cpt=2.5 (was 4)
Production data (N=354 cold-miss count_tokens probes, 2026-05-18..05-20):
real body-level chars-per-token is 1.17 median, 2.62 max — never near
the English-prose 4 baked into CHARS_PER_TOKEN. The gate has been
estimating text-token cost at 3.4× cheaper than reality and silently
rejecting every realistic Claude Code system slab.
Concrete production case (orig_chars=161,101, multi-col=2):
• OLD gate (cpt=4): text = 161101/4 = 40,275 tok
image = 8 × 5500 = 44,000 tok
→ reject (margin: -3,726)
• NEW gate (cpt=2.5): text = 161101/2.5 = 64,440 tok
image = 8 × 5500 = 44,000 tok
→ ACCEPT (margin: +20,440)
• Reality (cpt=1.17): text = 137,694 tok, savings = 93,694 tok/request
SLAB_CHARS_PER_TOKEN=2.5 is the upper bound of observed cpt across the
sample — picking the upper bound keeps the prime-directive guarantee
intact: the text-token estimate is a LOWER bound on real text cost, so
any `imageCost < textTokens` decision is also `imageCost < realTextCost`
for any future workload with cpt ≤ 2.5.
Scope: this is slab-specific. Reminders and tool_result content have
unknown shape (could be raw English prose with cpt~4), so those gate
call sites still use the conservative CHARS_PER_TOKEN=4. Host can
override per-request via TransformOptions.charsPerToken (e.g., to plug
a live empirical fit in front of the static default).
Tests: 2 new regression tests pinning (a) the production-shape 161k slab
compresses end-to-end without an explicit cpt override, and (b) the gate
math at cpt=2.5 vs cpt=4 produces accept-vs-reject on the same input.
272 tests pass, typecheck clean, build clean.
|
||
|
teamchong
|
e433b26caa |
feat(transform): lossless whitespace compactor on slab + reminder + tool_result
Production data (2026-05-20): real 161,101-char system slab being rejected by the row-aware profitability gate as `not_profitable`. multiCol=2 packs 282 rows/image, but 2000+ newline-bounded lines need ~10 images, and at 2500 tok × 10 × 1.10 = 27,500 tok the gate (correctly) refuses since the text baseline is cheaper. Fix: strip per-line trailing whitespace and collapse 3+ consecutive newlines to exactly 2 *before* feeding the renderer. Both transforms are lossless for the model (whitespace at line-end carries no signal; blank- line runs longer than 2 are visual noise). This drops visual-row count without changing meaning, flipping borderline-rejected slabs profitable. Accounting uses raw (pre-compaction) length for `origChars` and `compressedChars`, so the savings credit reflects what Anthropic would have billed — not the post-compaction figure. Applied at four sites: - combined static slab (system + tool docs) - system-reminder block - tool_result string content - tool_result array-of-blocks (joined text) 8 new unit tests for `compactSlabWhitespace`: empty input, trailing whitespace stripping, 3+ newline collapse, paragraph-break preservation, indent preservation, idempotency, realistic markdown shrink. 270 tests pass, typecheck clean, build clean. |
||
|
teamchong
|
59a416cb09 |
feat(history): always-on history compression + fix gate-vs-renderer asymmetry
Rip the compressHistory opt-in and run Variant C history-image collapse on
every request that has a closed-prefix run. There is now ONE codepath: no
toggle, no env switch, no per-request opt-in. The reason the option was
defaulted-off was a -250% live measurement on 2026-05-19 (commit
|
||
|
teamchong
|
178fd4af6f |
fix(gate): scale per-image token cost by numCols (close multi-col loss-risk)
Production runs multiCol=2 by default. The pre-flight gate
`isCompressionProfitable()` was using a flat TOKENS_PER_IMAGE=2500
constant at every numCols, so it believed multi-col images were ~2x
cheaper than they actually are and would let net-loss compressions
through on wider canvases.
Replace the flat constant with `effectiveTokensPerImage(numCols)`:
numCols=1: 2500 (empirical, N=33 cold-miss measurement)
numCols>1: 2500 * n * 1.10 (linear extrapolation + 10% margin)
The 10% margin absorbs extrapolation noise — we don't yet have
empirical cost measurements at numCols>=2. Per user constraint
"ok pre-flight gate is ok to be not correct as long as we don't
lose money," every uncertainty resolves in favor of pass-through.
At worst we miss opportunities; we never burn money on net-loss
images.
Test fixtures bumped to match the new break-even (~22k chars at
n=2). Two obsolete cpt-flip tests REWRITTEN to assert the safe
behavior: cpt override flips at numCols=1, but at numCols=2 the
safety margin holds and the slab stays rejected.
Old break-even at default cpt=4:
n=1: ~10k chars (1 image @ 2500 tok == 10k/4 = 2500 text tok)
n=2 [old/wrong]: ~10k chars (same flat cost)
n=2 [new/safe]: ~22k chars (1 image @ 5500 tok == 22k/4 text tok)
Verified: 263/263 tests pass, typecheck clean, build clean.
|
||
|
teamchong
|
dfd3bf514f |
feat(proxy): cache-aware baseline via second count_tokens probe
The baseline_tokens probe alone reports the COLD cost of the unproxied
body — but the user's request actually carries `cache_control` markers
that would have cached on the unproxied path too. Comparing cold-
baseline against warm-actual was an apples-to-oranges inflation of
saved%.
Fix: fire a SECOND count_tokens probe on the body TRUNCATED at the last
`cache_control` marker. Difference between the two is (cacheable_prefix,
cold_tail). Dashboard weights the prefix by the SAME cache class the
actual response landed in:
cacheable = baseline_cacheable_tokens or 0
cold_tail = baseline_tokens − cacheable
weight = cr>0 ? 0.10 (warm hit — both paths cache)
: cc>0 ? 1.25 (cold create — both paths build cache)
: 1.0 (no caching — marker rejected or absent)
baseline_input_eff = cacheable·weight + cold_tail·1.0
actual_input_eff = input + cache_create·1.25 + cache_read·0.10
saved = baseline_input_eff − actual_input_eff
Now the headline number is the exact unproxied counterfactual, not a
cold-every-time approximation. No estimation, no α, no assumptions —
every input is a measurement.
Wiring:
- src/core/proxy.ts: buildCountTokensCacheablePrefix() walks messages
backward → system backward → tools backward (reverse cache order) for
the last cache_control marker, truncates, attaches a sentinel user
message when needed. Second probe fires in parallel; null/missing
leaves the field absent.
- src/core/transform.ts + tracker.ts: new baselineCacheableTokens /
baseline_cacheable_tokens fields, emitted only when > 0.
- src/dashboard.ts: per-event cache-aware math in update() and
replay(); Totals renamed to baselineInputWeighted; /proxy-stats
payload field is `baseline_input_weighted`.
- src/sessions.ts: per-session tokensSavedEst uses the same formula so
session and global rollups agree.
Tests: 2 new in proxy-usage covering (a) body with cache_control →
two probes fire with different bodies and both numbers land on info,
(b) cacheable-prefix probe failure doesn't poison the full probe.
260→262 tests, all green; typecheck and build clean.
|
||
|
teamchong
|
f8902732bb |
feat(dashboard): real saved% from count_tokens, rip α-regression dead code
The headline saved%/saved$ numbers are now MEASURED, not estimated:
baseline = sum of /v1/messages/count_tokens(originalBody).input_tokens
actual = sum of (input + cache_create×1.25 + cache_read×0.10)
saved = baseline − actual
Output is excluded from both sides — it's identical with or without
compression (same prompt → same response), so it cancels in the
numerator and would only drag the percentage toward zero. The proxy
now makes one parallel side call to /v1/messages/count_tokens (free,
no billing) before forwarding /v1/messages, and the result lands on
TransformInfo.baselineTokens → TrackEvent.baseline_tokens → the
DashboardState rollup.
Removed (~280 lines of dead code):
- FALLBACK_ALPHA, FALLBACK_ALPHA_LOW, FALLBACK_ALPHA_HIGH constants
- estImageTokens(), baselineCost(), effectiveCost() helpers
- FitSample interface + FIT_SAMPLE_CAP + fitSamples ring buffer
- DashboardState.fitCosts() (~218 lines of OLS regression)
- Three-mode ladder (joint / constrained / stale) and all UI for it
- saved_pct_low/high, saved_effective_tokens_low/high range fields
- renderSavingsMath() range/regime branches
- m_pct_regime DOM node + its color/text wiring
- cost_fit payload field + all frontend consumers
/proxy-stats payload is now: requests, compressed_requests,
baseline_input_tokens, actual_input_weighted, saved_input_tokens,
saved_pct, saved_usd, pricing_assumptions, uptime_sec,
compression_enabled. Nothing else.
The recent-requests table now shows baseline / actual / saved per
row (input-side, matching the cards) instead of the old img-tok
estimate + effective_actual (which mixed output × 5 in inconsistently).
Sessions panel: per-session tokensSavedEst now uses the same real
baseline−actual formula. Events without both a count_tokens probe
AND an upstream usage block don't contribute to the rollup — no
synthetic 4-char/tok fallback.
Tests:
- dashboard-fit.test.ts: deleted (fitCosts is gone)
- proxy-usage.test.ts: inverted the privacy regression — count_tokens
is now explicitly expected as the one whitelisted side path; added
baselineTokens-attachment and count_tokens-failure-survives tests
- sessions.test.ts: rewrote the two savings tests against the new
baseline_tokens + usage formula
|
||
|
teamchong
|
1d46b72221 |
rip(proxy): remove count_tokens probe path — leaks original text
Sending the pre-compression body to /v1/messages/count_tokens means
Anthropic sees the original text, which is exactly what pixelpipe is
built to prevent. Probe results were 'free' in dollars but not in
privacy; the whole point of imaging the slab is that Anthropic does
NOT receive the slab text.
Removed:
- countTokensUpstream + buildCountTokensBody in src/core/proxy.ts
- measurePromise wiring + finalize await
- baselineTokensMeasured / actualTokensMeasured on TransformInfo
- baseline_tokens_measured / actual_tokens_measured on TrackEvent
- measuredEvents / effectiveCostActualMeasured / *Baseline*
fields on DashboardState
- measured_events / saved_pct_measured / effective_cost_*_measured
/ saved_effective_tokens_measured on /proxy-stats payload
- HTML headline branch + 'show calculation' rows for measured path
- 2 dashboard-fit tests asserting the measurement-path behavior
Added regression test: proxy must NOT hit any upstream path other than
/v1/messages.
Real-savings path now: self-derive α from billed usage + own char
walker. Coming next.
|
||
|
teamchong
|
c3506e3a2c |
fix(transform): flip compressHistory default to false
Round-3 measurement on 2026-05-19 with 128-turn history showed the feature loses money: 21K chars of text replaced with 140K tokens of imagery, net -250% savings per request on this traffic shape. The MARGINAL warning has been in the code comments since the feature shipped - it was opt-in via --history / COMPRESS_HISTORY=1 originally for exactly this reason. The recent flag-removal commit flipped it on as default without re-verifying the savings claim against current traffic. The math in baselineCost() correctly reports negative savings when imgTokensEst > txtReplaced, so the dashboard was honest about the loss - the bug is shipping the feature on by default before measuring a positive delta on representative traffic. Re-enable per-deployment via env var when measurement confirms a positive saved_pct on the specific shape. |
||
|
teamchong
|
a66a558b55 |
refactor: remove all CLI flags and behavior env vars — single codepath
The proxy used to expose ~20 behavior toggles (--no-compress, --no-tools, --no-schemas, --no-reminders, --no-tool-results, --history, --no-history, --history-keep-tail N, --history-min-prefix N, --min-chars N, --min-reminder-chars N, --min-tool-result-chars N, --cols N, --multi-col N, --no-track, --events-file PATH, --port N, --upstream URL) plus matching env vars. Each one was a dead branch in the common case and a configuration knob nobody had a principled reason to pick a value for. Replaced with: exactly one way to run the proxy. Every compression mode on, every tuning parameter at its measured-best value, history compression always active (was opt-in dead code). The only adjustable surface is deployment concerns — where to listen, what to forward, where to log — env-var only: PORT default 47821 ANTHROPIC_UPSTREAM default https://api.anthropic.com PIXELPIPE_LOG default ~/.pixelpipe/events.jsonl CLI accepts only --help and --version. Anything else exits with non-zero and a one-line error. Notable behavior change: - DEFAULTS.compressHistory flipped false → true. Variant-C history- image compression now runs on every request (was opt-in via --history / COMPRESS_HISTORY=1). The static-slab cache_control placement is on the system image, not on the history-replacement chain, so the published cache-topology risk does not apply. Touched: - src/node.ts: CliOpts → RuntimeConfig (3 fields). parseCli is now ~25 lines. printHelp reduced to env-var-only doc. - src/core/transform.ts: DEFAULTS.compressHistory = true. Comment updated. - scripts/restart.sh: drops PROXY_ARGS array entirely. Port read from PORT= env var. Unknown args rejected. - tests/restart.test.sh: replaces flag-passthrough test with a reject-unknown-args test. 4/4 pass. - tests/history.test.ts: renames the "compressHistory:false (default)" test to reflect the new always-on default; asserts the input-not- mutated invariant remains. - README.md: configuration table reduced to 3 env vars; restart examples drop CLI flag passthrough. Tests: 277/277. Typecheck + build clean. Restart script tests: 4/4. |
||
|
teamchong
|
c4aa5af342 |
refactor(proxy): remove measureSavings flag — count_tokens always runs
The measurement code was previously gated behind a measureSavings flag (--measure-savings CLI, PIXELPIPE_MEASURE_SAVINGS env var, optional ProxyConfig field). That's a knob nobody asked for and a second codepath whose only purpose is to leave the dashboard reporting estimates instead of ground truth. The flag is wrong for this codebase: - Single codepath is the convention. The proxy either measures or doesn't — there's no useful "sometimes" mode. - count_tokens is free, runs in parallel with the main upstream forward, and doesn't add user-visible latency. - An opt-in default means the headline dashboard number stays as an estimate forever in 99% of deployments — defeats the whole feature. Removed: - ProxyConfig.measureSavings field - CliOpts.measureSavings field - --no-measure-savings CLI handler - PIXELPIPE_MEASURE_SAVINGS env var - The if (config.measureSavings) wrapper in proxy.ts - Help-text entry - Stale comment references in tracker.ts, transform.ts, dashboard.ts The proxy now unconditionally fires count_tokens on both bodies for every /v1/messages request. Failure is silent — when count_tokens returns null the event still emits without the measured fields and the dashboard falls back to the regression estimate per existing fallback logic. Tests: 277/277. Typecheck + build clean. No behavioral test changes; the existing measurement-path test exercises the synthetic event shape directly, not the flag. |
||
|
teamchong
|
6f0d1a9914 |
feat(proxy): ground-truth saved_pct via count_tokens on both bodies
Replace estimation with measurement. Anthropic's /v1/messages/count_tokens
returns the exact tokenizer output for any request body, free of charge.
Calling it on the pre-transform body and the post-transform body gives
the EXACT delta in input tokens, with no α/β regression, no fallback
constants, no uncertainty band needed.
Opt-in (default off, since it adds two parallel HTTP roundtrips per
/v1/messages — count_tokens itself is free but the latency is real):
CLI flag: --measure-savings
Env var: PIXELPIPE_MEASURE_SAVINGS=1
When enabled the proxy fires both count_tokens calls in parallel with
the main upstream forward (no critical-path latency hit) and the event
is held for emit until both probes complete. Failures are silent — the
event still emits without the measured fields and the dashboard falls
back to the estimate.
Per-event new fields (events.jsonl):
baseline_tokens_measured exact pre-transform input tokens
actual_tokens_measured exact post-transform input tokens
Dashboard:
/proxy-stats gains: measured_events, effective_cost_actual_measured,
effective_cost_baseline_measured, saved_effective_tokens_measured,
saved_pct_measured. All null when no measured events exist; non-null
once ≥1 lands.
HTML headline switches to the measured number whenever measured_events
≥ 1. Subtitle reads "measured · count_tokens on N requests" to make
the source explicit. Falls back to the estimated range otherwise.
Measurement math:
delta_tokens = baseline_measured - actual_measured
bill_rate = (cc/total)·1.25 + (cr/total)·0.10 [same cache mix
as the actual call]
baseline_eff = actual_eff + delta_tokens · bill_rate
saved_pct_measured = (baseline_eff - actual_eff) / baseline_eff × 100
No α. No β. No FALLBACK_*. The numerator (delta_tokens) is the exact
output of Anthropic's tokenizer on both bodies; the multiplier is the
billed cache mix that already happened.
Tests: 275 → 277 (+2). Ground-truth path: 3 events with known token
deltas yield the expected saved_pct_measured. Fallback path: when no
event carries measurement, all saved_*_measured fields are null.
Typecheck + build clean.
|
||
|
teamchong
|
e27c19c2a3 |
feat(transform): live-α charsPerToken closes not_profitable max-savings leak
The break-even gate (isCompressionProfitable) used a hard-coded CHARS_PER_TOKEN=4 (Anthropic's English avg). Live Claude Code traffic has α≈0.5-0.9 (1.1-2 chars/tok) — JSON-dense tool docs, structured CLAUDE.md slabs, etc. Off-by ~3.5× on the text side at α=0.88. Production trace (events.jsonl, 2026-05-19): 200+ requests with system slabs 113K-170K chars hit reason="not_profitable" because the gate's textTokensEq = len/4 was a fraction of the row-aware image-token cost. Example event: slab=169_632 chars, cache_create_tokens=148_891. Gate's textEq=42K rejected vs imgCost=127K. With live α (1.14 ch/tok), textEq=149K → ACCEPT. We were leaving real cache_create money on the table on every one of those calls. - TransformOptions gains charsPerToken?: number. Default 4 (back-compat). - isCompressionProfitable() takes charsPerToken as a 5th optional param; defensive clamp on ≤0 / NaN / Infinity falls back to CHARS_PER_TOKEN. - All 4 internal gate call sites (static slab + reminder + tool_result string + tool_result array) thread o.charsPerToken through. - ProxyConfig.transform now also accepts a function: () => TransformOptions. Resolved per-request inside createProxy so the host can inject dynamic values. Static-object form unchanged for Workers / tests. - node.ts wraps the static transform options in a closure that asks dashboardState.fitCosts() each request and passes the live chars_per_token. Fit-null falls through to the baked-in default — no chicken-and-egg deadlock since normal-sized slabs still feed the fit ring even before the huge-slab cases get unblocked. - 4 new gate tests: stale-default rejects the 170K newline-heavy slab; live α=1.14 flips the same slab to profitable; defensive clamp on bogus charsPerToken; end-to-end transformRequest passes through. Tests: 269 → 273 (+4). Typecheck + build clean. |
||
|
teamchong
|
17b77c7492 |
fix(transform): count tools[], tool_use, thinking blocks in outgoingTextChars
The walker was missing three big text-bearing paths the upstream tokenizer sees: top-level `tools[]` (name + description + JSON schema), `tool_use` blocks (name + serialized input), and `thinking` blocks. Under-counting these inflates α in `tokens ≈ α·textChars + β·pixels`, which biases the dashboard's `saved_pct` HIGH — the honesty cost HANDOFF item #1 called out. - Walker now sums: system + tools[].{name,description,input_schema} + per-block text/tool_use.input/tool_result.{tool_use_id,content}/ thinking. JSON.stringify via safeStringifyLen() for object fields so the count matches what the API actually serializes. - Lift the walker call out of the compression-success path: also run it on the below_min_chars and not_profitable early returns. Without this, α was sampled only from "big enough to compress" requests. - New `outgoingTextChars walker (denominator honesty)` describe block in render.test.ts: 7 tests covering baseline, tools[], tool_use, tool_result (string + array forms with image skip), thinking, unknown-block skipping, and a JSON.stringify(req).length upper bound asserting walker ≥ 50% of envelope on a content-dense request. - Confirms walker stays strictly below the JSON envelope (structural keys/braces still excluded — that's intentional). Tests: 254 → 261 (+7). Typecheck + build clean. |
||
|
teamchong
|
937f0f8817 |
prune: drop dead 'placement' + 'compressSystem' knobs (system-field branch was API-rejected)
The `placement === 'system'` branch attached image blocks to the `system`
field, which Anthropic 400s with `system.N.type: Input should be 'text'`.
The existing CLI help text already warned about this. No tests exercised
the dead branch (grep for `placement.*'system'` finds zero call sites).
`compressSystem` was only consulted inside that same dead branch, so it
went out with it. The other granular `compress{Tools,Schemas,Reminders,
ToolResults}` flags + the `compress` master switch stay — they're useful
debugging toggles with active call sites.
Touched:
- transform.ts: drop both fields from TransformOptions + DEFAULTS;
collapse the if/else around system-rewriting into a bare block (no
re-indentation noise, no behavior change).
- node.ts: drop --placement CLI flag, PLACEMENT env, help text entries,
startup log column, and the TransformOptions wire-up.
- worker.ts: drop PLACEMENT from Env + transform options.
- render.test.ts: refresh three comments that referenced the now-gone
"default placement is 'user'" language; the test bodies were already
correct and didn't reference the removed knobs.
20-line net reduction, 241 tests pass, typecheck + build clean.
|
||
|
teamchong
|
f1688e9f46 |
transform+render: R2 multi-col default-on + per-request pixel/char instrumentation
Two pieces of work landing together because they're interleaved in transform.ts and need to ship at the same time for the dashboard fit-wiring in the next commit to work. # R2 multi-column rendering (default `multiCol=2`) Single-col packing wastes ~68% of Anthropic's 1568×1568 image budget on real Claude Code tool docs (~38 chars/row). The row-aware break- even gate correctly rejects compression on those slabs — proxy ends up doing nothing on the heaviest requests. Multi-col packs N text columns side-by-side per image so each image covers `N×LINES_PER_IMAGE` wrapped lines, dropping image count by ~N and crossing break-even. - render.ts: new `renderTextToPngsMultiCol(text, cols, numCols)` with column-major layout (col 0 fills top-to-bottom, then col 1, …). Plus `multiColWidth` / `maxFittingCols` helpers. CLI auto-clamps numCols so the canvas can never exceed the 1568 px width cap. - transform.ts: `multiCol` option threaded through; gate, image-count estimator, paging budget, and all render paths take `numCols`. Intro text now describes column reading order when numCols>1 so the OCR pass doesn't read row-wise across columns. - Host wiring: node.ts (--multi-col / MULTI_COL=N, default 2), worker.ts (MULTI_COL env, default 2). Startup log shows multi_col=N. - numCols=1 stays a byte-identical passthrough to renderTextToPngs so flipping back can't regress the cache_control hit rate. # Empirical-cost instrumentation The dashboard's `OPUS_IMAGE_TOKEN_COST = 2500` and `CHARS_PER_TOKEN = 4` are stale pre-Opus-4.7 constants. Fact-checked: 4.7 changed the TEXT tokenizer only (~1.20× inflation on English, ~1.36× on TypeScript); image cost is unchanged for same-size images. Direction of error in the dashboard headline: it UNDERSTATES text savings. Ditched the "guess from one cold-miss" approach in favor of real instrumentation that pins both axes at once: - TransformInfo.imagePixels — Σ width×height across every image this request (static slab + reminders + tool_results + history collapse). Threaded through textToImageBlocks, collapseHistory. - TransformInfo.outgoingTextChars — every char of TEXT remaining in the transformed outgoing body, computed in one walk before serialization (countOutgoingTextChars). - Tracker event schema picks up image_pixels + outgoing_text_chars. - history.ts: HistoryCollapseInfo.collapsedImagePixels accumulates. The regression `tokens ≈ α·outgoing_text_chars + β·image_pixels` over N cold-miss events solves for empirical chars/token (α) and pixels/token (β) under the live model — no assumed constants. # Tests - 8 new render tests covering the multi-col passthrough contract (numCols=1 byte-identical to renderTextToPngs), canvas dimensions, image-count halving, determinism, charsRendered sum, estimator parity, CLI clamp, and CJK wide-glyph wrap. - paging.test.ts "counts multiple tool_results" threshold loosened from >770k to >700k to cover both single-col and multi-col budgets at the same maxImagesPerToolResult cap. - All 241 tests pass, typecheck + build clean. |
||
|
teamchong
|
aed76dbdc2 |
transform: compact JSON in tool docs (kills the sparse-fill crisis)
renderToolDoc pretty-printed every input_schema with 2-space indent,
putting each property on its own line. At cols=100 this wasted ~70% of
horizontal space — schemas rendered to many short rows instead of a few
dense ones.
Live measurement on 2026-05-19: 150 KB of pretty-printed tool docs in a
177 KB combined slab → 31 static-slab images per request at 40% fill.
With the new row-aware break-even gate (
|
||
|
teamchong
|
f0176c1432 |
transform: row-aware break-even gate stops net-loss compressions
isCompressionProfitable previously estimated image count as
ceil(chars / charsPerImage), which assumes uniform full-width line-fill.
renderTextToPngs actually budgets by wrapped visual rows (141 per image),
so newline-heavy code/logs and short-line content render to many more
images than the chars-based estimate predicts.
Live impact: dashboard reported -69% reduction (net loss). 33 compressions
averaged 5494 chars/image (39% fill ratio) — gate under-counted cost by
~9× on sparse content and let net-losing compressions through.
Fix:
- isCompressionProfitable now accepts the full text string; uses
estimateImageCount (the same row-aware math renderTextToPngs uses).
- Adds an optional imageCountCap for paths that truncate before rendering
(tool_results). Cost bounded by cap; savings still measured against
the full pre-truncation length, so 500KB logs still profit.
- All 4 production call sites in transform.ts pass the string. The
tool_result sites also pass maxImagesPerToolResult as the cap.
- Number-arg form retained for back-compat with the existing
isCompressionProfitable(N) unit tests.
Tests:
- 3 new unit tests pin the row-aware regression: sparse content rejected,
dense content accepted, capped content (paging) accepted.
- 7 integration tests previously used 'claude.md\n'.repeat(5000) — sparse
fixtures the new gate correctly rejects. Updated to dense fixtures
('x'.repeat(40000) or single-line repeats without embedded newlines)
so they still exercise the cache_control / determinism / reminder /
tool_result code paths they're meant to.
|
||
|
teamchong
|
8fbd53ea3e |
dashboard: fix bogus savings math + emit compressed_chars
The dashboard headline was showing 0% saved because the math was
apples-to-oranges:
- orig_chars = static slab + tool docs ONLY (~135 KB)
- image_count = ALL images including reminders + tool_results (~33)
- txtReplaced = 135k/4 = 33.7k tokens
- imgTokensEst = 33 × 2500 = 82.5k tokens
- extraText = max(0, 33.7k - 82.5k) = 0 ← clamped negative
- baseline == paid → 'saved 0%' headline despite real savings happening
Two fixes:
1. Track 'compressedChars' separately from 'origChars'. It accumulates
only when a block actually image-renders (passes the break-even gate
AND any per-block min thresholds). Includes the static slab + all
compressed reminders + all compressed tool_results.
2. Remove the Math.max(0,...) clamp in baselineCost(). Negative savings
surface honestly — they're the signal that small-block compressions
are cost-bleeding, which the break-even gate is supposed to prevent.
Wires:
- TransformInfo.compressedChars (new field, mandatory).
- All 4 image-render success paths accumulate (static slab + reminder
+ tool_result string branch + tool_result array branch).
- TrackEvent.compressed_chars (new optional field, emitted when > 0).
- dashboard.baselineCost() takes compressedChars now; clamp removed.
- sessions.aggregateSessions() also uses compressed_chars + honest math.
229/229 tests passing. Bundle stays at 496 KB gzipped.
Post-restart, the dashboard 'tokens saved' will show real numbers
including negatives. Empirically expect ~1-5% net positive in
steady-state cache-hit traffic.
|
||
|
teamchong
|
97fdee8421 |
history: Variant C closed-tool-sequence collapse + honest per-session savings
Two distinct improvements in one slice:
1. History-image compression (Variant C — opt-in, default OFF).
New src/core/history.ts module:
- findClosedPrefixBoundary(): walks tool_use_id openSet, returns largest
fully-closed prefix index. Mid-flight tool_use blocks bail the collapse.
- blocksToText(): linearises content blocks for the history image. Drops
thinking blocks (per Opus 4.7+ docs: only most-recent must be bit-perfect),
renders tool_use args + tool_result inner text, emits [image] placeholder
for nested images.
- collapseHistory(): orchestrates. Applies isCompressionProfitable() gate.
Returns a synthesized {role:'user', content:[text,image,text]} prepended
message + the elided turns dropped from messages[].
New TransformOptions: compressHistory (default false), historyKeepTail (4),
historyMinPrefix (10). Wired through src/node.ts + src/worker.ts as
COMPRESS_HISTORY / HISTORY_KEEP_TAIL / HISTORY_MIN_PREFIX env vars.
Cache_control: history-image carries NO cache_control on this first cut.
Static slab keeps its 1 breakpoint. Conservative — telemetry will reveal
whether the cache-miss rate on history images warrants the round-3
rebalance in a follow-up slice.
29 new tests in tests/history.test.ts: closed-prefix detection (parallel
tool calls, mid-flight, openSet straddling), block linearisation, full
collapse flow, isCompressionProfitable gate firing on small histories.
2. Per-session savings: honest formula (fixes task #63).
Previous src/sessions.ts computed saved = orig_chars - image_bytes — a
nonsense comparison of chars to PNG bytes, then clamped via Math.max(0,...).
Result: charsSaved/tokensSavedEst was 0 for every session including ones
with hundreds of MB of compressed traffic.
New formula mirrors src/dashboard.ts:
textTokens = orig_chars / 4
imageTokens = image_count × 2500
tokensSaved = textTokens − imageTokens (CAN be negative)
Negatives are NOT clamped — they surface the cost-bleed on small-block
compressions that the per-block break-even check is supposed to prevent.
2 new tests: positive-net case, all-negative case (must report negative).
229/229 tests passing. Bundle ~500 KB gzipped.
|
||
|
teamchong
|
04b1526bd5 |
transform: large-tool_result paging + adaptive break-even derivation
Two related improvements:
1. Per-tool_result paging / truncation. When a single tool_result would
render to more than maxImagesPerToolResult images (default 10), the
source text is truncated head+marker+tail BEFORE rendering. Classifier
picks the truncation strategy:
- structured (JSON/YAML/diff): head-only, tail is repeating elements
- log (timestamps / [LEVEL] prefixes): 60% head + 40% tail
- other (prose, dumps): same head/tail split
The marker tells the model what was elided so it can reason about the
gap: '[ pixelpipe paging: omitted N lines (K chars) ... ]'. Keeps
requests under Anthropic's 100-image-per-request cap even when a single
tool dumps a multi-megabyte log.
2. Adaptive CHARS_PER_IMAGE. Previously hardcoded at 14_100 (Unifont 10px
@ cell 5×11). Now derived at module load from ATLAS_CELL_H + render.ts
constants. When gen-atlas regenerates with a different font/size, the
break-even threshold automatically updates: at Cozette 4×7 the chars/
image jumps from 14.1k → 22.2k and the break-even drops from ~10k → ~3k.
Single source of truth in src/core/.
Wires:
- TransformOptions.maxImagesPerToolResult (default 10)
- TransformInfo.truncatedToolResults + omittedChars
- TrackEvent.truncated_tool_results + omitted_chars (emit when > 0)
- Two new exported helpers: estimateImageCount, classifyContent, truncateForBudget
- Paging fires before image-encoding at both tool_result branches
(string content and array-of-text-blocks content)
Tests: +28 paging-specific (paging.test.ts), 199/199 total green.
Bundle delta: negligible (~3 KB raw, ~1 KB gzipped). 488 KB dist gzipped.
Closes the paging-implementer worktree merge that was deferred earlier.
|
||
|
teamchong
|
e7489ca1fb |
transform: per-block compression break-even check + honest dashboard
URGENT production-cost fix. Empirical data (orig_chars=169779, image_count=88)
showed we were paying ~220k image tokens to replace ~42k text tokens — a
net LOSS of ~178k tokens per request. Dashboard hid this via Math.max(0,...).
Two changes ship together:
1. isCompressionProfitable(textLen) gate in transform.ts. Before any
per-block image encode (static slab, reminder, tool_result), compute:
estImages × TOKENS_PER_IMAGE vs textLen / CHARS_PER_TOKEN
Compress only when the image side wins. Constants:
CHARS_PER_IMAGE = 14100 (Unifont 10px, cell 5×11, 100 cols)
CHARS_PER_TOKEN = 4
TOKENS_PER_IMAGE = 2500 (empirical, history-researcher round 3)
Result: blocks under ~10k chars stay as text. Static slab still
compresses (always above break-even).
2. Coarse thresholds bumped as fast-path:
minReminderChars 1000 → 5000
minToolResultChars 5000 → 10000
(Cover the obvious-no cases without invoking the precise check.)
3. Dashboard honesty: removed Math.max(0,...) clamp in baselineCost().
Negative savings now display as negative (no more hidden cost-bleed).
4. Telemetry: passthrough_reasons.below_threshold + .not_profitable
counters surface how often each gate fires.
After restart, dashboard 'saved' should jump from clamped-0 to a real
positive number (~1-3% on aggregate; static slab compression is the
only block consistently profitable at our current cell).
|
||
|
teamchong
|
f199640bdc |
ship Unifont @ 10px multilingual rendering + 400 diagnosis fixes
- Switch font: JetBrains Mono @ 15px → Unifont 16.0.04 @ 10px. Sparse atlas with double-width EAW dispatch for CJK. Full BMP profile (35,501 glyphs) covers Latin/Cyrillic/Greek/Hebrew/Arabic/CJK/Hangul plus Letterlike, Misc Technical, Block Elements, Geometric Shapes, Misc Symbols, Dingbats, Enclosed Alphanumerics, all box-drawing. - Fix cell-height bug (was hardcoded 9; now probed from font metrics via Math.ceil(maxAscent+maxDescent)). Previously clipped the top 2 rows off every CJK glyph since the original Unifont ship. - Expand tabs to spaces with 4-column tab stops, EAW-aware. Was the silent-drop source for 99.6% of dropped chars in production. - Add dropped_codepoints_top telemetry (top 20 per event, only emitted when drops > 0) so future coverage gaps surface in events.jsonl. - Raise minToolResultChars 2000→5000 and minReminderChars 1000→2000 to stop net-losing tokens on small compressions, per ~2,500 tok/img empirical measurement (N=33 cold-miss events). - Fix dashboard per-image-cost formula: was pngBytes/375 (~190 tok), now imageCount * 2500 (the measured empirical rate). Reduces the inflated "tokens saved" headline to honest numbers. - Schema-stub fix in compressSchemas path: preserveSchemaShell() retains type / properties / required / enum / oneOf / anyOf / $ref / numeric constraints recursively (depth 20), strips only descriptions and metadata. Prevents 400 errors on first-tool-call requests. - Capture full gzipped 4xx body + per-event req_body_sha8 for upstream error diagnosis. Sidecar to ~/.pixelpipe/4xx-bodies/ when over 32 KB inline cap. Workers-safe via CompressionStream + Web Crypto. - scripts/restart.sh: pnpm-aware one-command kill+rebuild+start with graceful SIGTERM→SIGKILL escalation, build-failure abort, port-in-use precheck, --no-build flag for fast iteration. Shell-tested. - Bundle 1.58 MB gzipped on Node. 91/91 vitest + 4/4 shell tests. |
||
|
teamchong
|
be372a0c3a |
legacy: drop python reference oracle (live savings confirmed at 98.7%)
HANDOFF item #4 was "Delete legacy/python/ — but only after #1 confirms live savings." Item #1 (live cache-rate validation) now passes: over 232 compressed POST /v1/messages calls in a 46-minute real-traffic window, cache hit rate by tokens is 98.7%, clearing the HANDOFF 60% bar by 38 points. The Python reference has done its job. Removes (1831 deletions): legacy/python/proxy.py (975) full Python proxy implementation legacy/python/diff_transform.py (213) body-diff parity oracle legacy/python/cli.js (170) shim legacy/python/visual_compare.py (150) PNG-diff parity oracle legacy/python/diff_render.py (136) render-only parity oracle legacy/python/gen_atlas.py ( 75) atlas generator (TS port owns this now via scripts/gen-atlas.ts) legacy/python/install.js ( 50) installer shim legacy/README.md ( 35) Comment / doc updates to remove references: README.md drops the "## Legacy" section CLAUDE.md drops the "## Parity oracle" section + tree entry src/core/transform.ts header comment no longer cites legacy/python src/dashboard.ts header comment notes the formulas were ported, not where from docs/TRANSFORM_INFO.md edits land in the separate TRANSFORM_INFO commit The byte-output determinism the Python oracle existed to guard against is now pinned by tests/render.test.ts ("renders identical input to byte- identical output (determinism = cacheability)") and the systemSha8 fingerprint in every JSONL row. If a future change ever breaks determinism, the test will catch it before merge, and the production-deployed cache hit rate will catch it within minutes after. Gates: typecheck clean, 38 tests pass, build clean dist/node.js. |
||
|
teamchong
|
e219b37cd5 |
transform: KNOWN_STATIC_TAGS allowlist (silence <types> canary)
The canary in splitStaticDynamic flags tag-shaped blocks in the static slab that aren't in DYNAMIC_BLOCK_TAGS — designed to catch a new Claude Code release that ships a per-turn tag we'd accidentally bake into the cached image. In production it has been firing on `<types>` for 93% of real requests (125 of 134 in the user's events log), which is noise: `<types>` is part of Claude Code's built-in tool documentation, not a per-turn rotation. Add a second list, KNOWN_STATIC_TAGS, that mirrors DYNAMIC_BLOCK_TAGS in shape. The unknown-tag computation now subtracts BOTH lists, so a tag is only "unknown" when it's in neither the per-turn set nor the known-static set. Seed KNOWN_STATIC_TAGS with ['types']. Updates the warn log in src/node.ts to mention both options so a reviewer of a NEW unknown tag knows which list to extend. Adds a regression test pinning that <types> in the static slab no longer produces info.unknownStaticTags. |
||
|
teamchong
|
a5f61ba6ee |
transform: compress large tool_result text content across user messages
Tool results (Bash output, Read file contents, etc.) accumulate as the
session grows. Once produced they're static — same bytes turn after turn
— but the model gets billed for them on every call until they fall out
of cache. Image-rendering them at the source compounds the savings
across the whole conversation.
Walks ALL user messages (not just the first — that's the reminder-
compression scope) and rewrites tool_result blocks whose content is at
or above minToolResultChars (default 2000, matching legacy/python/proxy.py):
- content: string → replace with image block(s) (whole content goes)
- content: array → walk inner blocks, replace each TextBlock ≥ threshold
with image blocks; leave ImageBlocks and short text
alone
- is_error: true → leave the whole block alone. Anthropic rejects
images inside is_error tool_results
(`tool_result with is_error=true cannot contain images`).
Like reminder compression these images carry NO cache_control — the
system+tools image is our only breakpoint. The savings are 65-73% of
input tokens per compressed block on Opus 4.7.
Wire-up:
- src/core/transform.ts: TransformOptions.compressToolResults (default
true) + minToolResultChars (default 2000). TransformInfo.toolResultImgs
counter. Reuses textToImageBlocks / approxBlockBytes from commit 3.
- src/core/tracker.ts: TrackEvent.tool_result_imgs flows from the
transform info.
- src/worker.ts: COMPRESS_TOOL_RESULTS + MIN_TOOL_RESULT_CHARS env vars.
- src/node.ts: CliOpts fields, parseCli cases (--no-tool-results,
--min-tool-result-chars), help text, env-var summary, `tr+N` in the
per-request console line when tool_result_imgs > 0, and tool_results=
in the startup config summary.
Tests:
- "compresses large tool_result text content across user messages"
confirms info.toolResultImgs ≥ 1, the tool_result block's content
becomes an array of image blocks, and those images carry no
cache_control.
- "leaves is_error tool_results untouched (Anthropic forbids images
there)" confirms info.toolResultImgs is 0 and the content stays a
plain string when is_error: true.
|
||
|
teamchong
|
40037568e2 |
transform: compress <system-reminder> blocks in the first user message
Claude Code re-injects the same long <system-reminder> blocks (CLAUDE.md
hints, task-tools reminders, the dynamic skills list, etc.) into the
FIRST user message on every turn. They're text, so they're not part of
the system+tools image we already cache — they hit the wire fresh every
turn. This rewrite renders any reminder block ≥ minReminderChars (1000
by default, matching legacy/python/proxy.py) to PNG images placed inside
the user message content array, just like the static slab.
Why: the system+tools image is the only cache_control breakpoint we're
allowed (Anthropic caps at 4 and Claude Code uses the others). Per-block
reminder images therefore carry NO cache_control — they're per-turn
content that the model OCRs without prefix-cache benefit. The win is
token cost, not cache: reminders are 65-73% cheaper as PNG than as text,
and a typical turn has 1-3 of them.
Layout inside the first user message after compression:
[intro text] ← static (helps OCR framing)
[image block(s)] ← static; LAST has cache_control
↑ cache breakpoint (the only one)
[End of rendered context.] ← static text closer for the image
[processed existing content] ← per-turn (incl. reminder images)
Wire-up:
- src/core/transform.ts: new TransformOptions.compressReminders (default
true) + minReminderChars (default 1000). Two new helpers,
textToImageBlocks() (wraps renderTextToPngs + makeImageBlock with
no cache_control) and approxBlockBytes() (b64 → byte-count for the
imageBytes telemetry, no second base64 round-trip). The placement
='user' branch now walks `existing` and rewrites long reminders.
- src/core/tracker.ts: TrackEvent.reminder_imgs flows from
TransformInfo.reminderImgs through toTrackEvent so `pixelpipe stats`
and the dashboard can attribute compression contributions.
- src/worker.ts: COMPRESS_REMINDERS + MIN_REMINDER_CHARS env vars.
- src/node.ts: matching CliOpts, parseCli cases (--no-reminders,
--min-reminder-chars), help text, env-var summary, and `rem+N` in
the per-request console line when reminder_imgs > 0.
Tests:
- "compresses long <system-reminder> blocks in the first user message"
confirms info.reminderImgs ≥ 1, the reminder text is gone from
user-content text blocks, the user's actual prompt survives, and the
new reminder image blocks carry NO cache_control.
- "leaves short <system-reminder> blocks alone (below minReminderChars)"
confirms info.reminderImgs is 0 and the reminder text passes through
as text.
|
||
|
teamchong
|
ba13bf3a42 |
dashboard: live HTML + JSON endpoints in the Node host
Ports the live dashboard from legacy/python/proxy.py into TypeScript.
Node-only by design — the Workers host doesn't expose a dashboard (use
Workers Logs / Logpush for that).
New module src/dashboard.ts:
- DashboardState keeps a 50-entry ring buffer of recent requests, plus
running totals for the savings-vs-baseline computation (same formula
as the Python: input + cache_create*1.25 + cache_read*0.10). One
latest PNG is held in memory for the preview pane.
- Replay-on-startup: reads the tail of ~/.pixelpipe/events.jsonl so the
UI doesn't show an empty table after a process restart. Cumulative
totals are NOT replayed (file may have rotated; double-counting beats
starting from zero badly here? — no, starting fresh is the lesser
evil after rotation, hence "best-effort replay of the ring buffer
only").
- Four HTTP handlers: serveHtml (the inline dashboard page),
serveStats (JSON aggregate), serveRecent (JSON ring buffer),
servePng (the latest preview image). dashboardPath() classifies a
path so the Node server can intercept before the proxy handler.
- Inline HTML uses CSS object-position to crop the preview at the
image's native resolution — no server-side PNG decode needed.
Plumbing changes:
- src/core/transform.ts: capture info.firstImagePng / firstImageWidth /
firstImageHeight after rendering. NOT persisted to JSONL; toTrackEvent
drops them. Memory cost is bounded (only the latest image survives,
the dashboard replaces on every update).
- src/node.ts: import DashboardState, instantiate, seed via replay,
call dashboard.update(e) inside onRequest BEFORE tracker.emit (the
tracker strips firstImagePng). The HTTP handler intercepts the four
dashboard paths and answers from DashboardState; everything else
falls through to the proxy.
- src/node.ts / src/worker.ts: bump the CLI/env default for PLACEMENT
from 'system' to 'user' so the runtime path matches transform's
default (
|
||
|
teamchong
|
e0950efa6f |
transform: default to placement='user' and stamp ttl='1h' on image cache_control
Two protocol-correctness fixes that have to ship together. 1. **Default placement: 'system' → 'user'.** Anthropic's `system` field is typed as `string | TextBlock[]` — image blocks there come back as `400 system.N.type: Input should be 'text'`. Images must go into a user-message content array. 2. **ttl='1h' on the image's cache_control.** Anthropic enforces the rule "ttl='1h' must not come after ttl='5m'" in processing order (tools → system → messages). Claude Code marks its own user-message content with ttl='1h'; if we leave ttl unset it defaults to '5m', our block lands first in processing order, and the request 400s the moment Claude Code's own 1h breakpoint shows up later in the conversation. The existing image-placement tests now look in messages[0].content instead of out.system, and a new test pins the ttl='1h' invariant directly. |
||
|
teamchong
|
102a0966f5 |
transform: add determinism test + unknown-tag canary
Two cheap guardrails based on architectural review: 1. Determinism test: renders the same input twice and asserts byte equality of the produced PNGs. Without this guarantee, identical system prompts on consecutive turns would yield different image bytes, silently destroying the cache hit rate that justifies the whole proxy. The test currently passes — locking it in so a future refactor can't regress it without noticing. 2. Unknown-tag canary: splitStaticDynamic now sniffs the *static* slab for any <tag>...</tag>-shaped block whose name isn't in DYNAMIC_BLOCK_TAGS. Surfaced via info.unknownStaticTags and emitted on the tracker event as unknown_static_tags. If Claude Code ships a new per-turn tag in a future release, it'll show up in logs within hours instead of being silently baked into the cached image (where it would tank cache hit rate with no error). Also adds tracker.ts (Tracker interface, TrackEvent shape, JsonLogTracker, noopTracker, toTrackEvent normalizer). Hosts will wire these in the next commit. |
||
|
teamchong
|
99559c463b |
transform: add sha8 fingerprints (system/claude_md/first_user)
Three short hashes carried in TransformInfo: - systemSha8: sha256[0..8] of the actual image-bound text. Stable across turns proves cache_control is doing its job. If it drifts unexpectedly, that's a cache leak to investigate. - claudeMdSha8: hash of just the CLAUDE.md slab (heuristic header detection). Lets us bucket requests by project even when cwd is absent or anonymized. - firstUserSha8: hash of first user message (4 KiB cap). Rough thread id since the wire protocol has none. All async via Web Crypto so Node + Workers share one impl. Hashing is in parallel via Promise.all; ~0.1ms overhead. +2 tests (systemSha8 invariant under dynamic-block change, firstUserSha8 format). |
||
|
teamchong
|
364373618f |
transform: extract env fields (cwd, branch, platform, today) into info.env
New EnvFields type and extractEnvFields() helper. Parses the <env> block Claude Code injects, plus 'On branch' / 'Branch:' lines that appear in <git_status> or context blocks. Read-only — telemetry signal only, never mutates the request. Per-project compression ratios, repeat-CWD detection, today's-date drift checks all become possible from logs now. +2 tests (full env block extraction, no-env-no-info). |
||
|
teamchong
|
84b4a3a59a |
transform: split static/dynamic system text for cache stability
Claude Code injects per-turn blocks (<env>, <context>, <git_status>,
<directoryStructure>, <system-reminder>) into the system prompt.
Previously those got rendered into the image alongside the static
slab (CLAUDE.md, agent defs, tool docs), which kills Anthropic's
prompt cache: image bytes drift turn-to-turn → 0% cache hits.
Now we split:
- static (cacheable) → image with cache_control: ephemeral
- dynamic (per-turn) → plain text block AFTER the image
- billing line → also AFTER the image (was BEFORE, also a
cache-killer — bug fix as a side effect)
Net effect: stable cache key on the big slab, model still sees cwd /
branch / today's date. Removes the need for users to pass
--exclude-dynamic-system-prompt-sections to claude.
TransformInfo now carries staticChars / dynamicChars / dynamicBlockCount
for downstream telemetry.
Tests: 8 existing pass unchanged + 3 new (env survives as text after
image, cache_control on image only, all-dynamic input is pass-through).
|
||
|
teamchong
|
f91c149d05 |
Port to TypeScript with dual Node + Cloudflare Workers targets
Runtime is pure Web Standard APIs (fetch, Request, Response, Uint8Array,
CompressionStream, crypto.subtle, btoa) — zero runtime dependencies.
Core (src/core/, runs identically on Node 18+ and Workers):
- atlas.ts auto-generated 9x15 JBM glyph atlas, base64-inlined (17KB)
- png.ts minimal grayscale PNG encoder via CompressionStream
- render.ts text → packed PNG, soft-wraps at 100 cols, ≤1568px tall
- transform.ts request body rewriter (system + tool docs → image blocks)
- proxy.ts fetch-handler that transforms then forwards to Anthropic
- types.ts Anthropic Messages API types we touch
Adapters:
- src/node.ts node:http server + CLI flag parsing
- src/worker.ts export default { fetch } for wrangler dev/deploy
Build tooling:
- scripts/gen-atlas.ts @napi-rs/canvas → atlas.ts (build-time only)
- scripts/build.mjs esbuild Node bundle; wrangler handles Worker
- tsconfig.json strict, ES2022, Workers types
Tests (vitest): 8 passing — PNG signature, base64 round-trip, single +
multi-image renders, transform no-op + compress paths, billing-line strip,
tool fold + stub.
E2E smoke: 16K char system → 2 PNGs (36KB) → mock upstream, 34ms.
Next: replace docs, then verify byte-output parity against legacy/python.
|