mirror of
https://github.com/teamchong/pxpipe.git
synced 2026-07-22 02:02:51 +02:00
d02a0f9baff149a73ea899eb50db77dc0cd48e09
51 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 teamchong
|
d02a0f9baf |
feat(dashboard): fold output tokens into saved_pct (full-bill framing)
effectiveCost() previously summed only input + cache_create·1.25 + cache_read·0.10, leaving output tokens out of the cost total. The headline saved_pct was therefore "% saved on the input-side bill" rather than "% saved on the full upstream bill." Output is small in absolute terms on typical Claude Code traffic (≈0.4% of cost in the sample events.jsonl), but the framing is what matters. Changes: - effectiveCost gains an outputTokens parameter and sums output × OUTPUT_TOKEN_RATE (=5.0, the Opus/Sonnet output:input rate ratio). - Totals fields renamed effectiveInput* → effectiveCost* to match the new full-bill semantics. Same on the JSON wire: effective_input_* → effective_cost_*. - update() now also passes the event's output_tokens into effectiveCost so both actual and baseline grow consistently. - replay() effective_actual computation includes output_tokens too, so historical rows align with the live formula. - Headline subtitles updated to "share of total bill saved" and "effective tokens (full bill)" so the unit is self-documenting. Math invariant: output is identical in actual and baseline (the model produces the same response regardless of prompt compression), so it cancels in the numerator saved = baseline - actual but inflates the denominator. saved_pct therefore moves SMALLER and more conservative when output is non-trivial — honest whole-bill terms. New test pins the invariant: two identical input-side runs with 100× different output_tokens give the SAME saved_effective_tokens but strictly different saved_pct, with the higher-output run lower. Tests: 273 → 274 (+1). Typecheck + build clean. |
||
|
teamchong
|
e27c19c2a3 |
feat(transform): live-α charsPerToken closes not_profitable max-savings leak
The break-even gate (isCompressionProfitable) used a hard-coded CHARS_PER_TOKEN=4 (Anthropic's English avg). Live Claude Code traffic has α≈0.5-0.9 (1.1-2 chars/tok) — JSON-dense tool docs, structured CLAUDE.md slabs, etc. Off-by ~3.5× on the text side at α=0.88. Production trace (events.jsonl, 2026-05-19): 200+ requests with system slabs 113K-170K chars hit reason="not_profitable" because the gate's textTokensEq = len/4 was a fraction of the row-aware image-token cost. Example event: slab=169_632 chars, cache_create_tokens=148_891. Gate's textEq=42K rejected vs imgCost=127K. With live α (1.14 ch/tok), textEq=149K → ACCEPT. We were leaving real cache_create money on the table on every one of those calls. - TransformOptions gains charsPerToken?: number. Default 4 (back-compat). - isCompressionProfitable() takes charsPerToken as a 5th optional param; defensive clamp on ≤0 / NaN / Infinity falls back to CHARS_PER_TOKEN. - All 4 internal gate call sites (static slab + reminder + tool_result string + tool_result array) thread o.charsPerToken through. - ProxyConfig.transform now also accepts a function: () => TransformOptions. Resolved per-request inside createProxy so the host can inject dynamic values. Static-object form unchanged for Workers / tests. - node.ts wraps the static transform options in a closure that asks dashboardState.fitCosts() each request and passes the live chars_per_token. Fit-null falls through to the baked-in default — no chicken-and-egg deadlock since normal-sized slabs still feed the fit ring even before the huge-slab cases get unblocked. - 4 new gate tests: stale-default rejects the 170K newline-heavy slab; live α=1.14 flips the same slab to profitable; defensive clamp on bogus charsPerToken; end-to-end transformRequest passes through. Tests: 269 → 273 (+4). Typecheck + build clean. |
||
|
teamchong
|
9c860f02dd |
feat(dashboard): sessions bulk select + delete (Gmail/GitHub pattern)
The sessions table previously only supported per-row deletion via the "del" button — no way to clear many at once. Add an industry-standard multi-select pattern. Pattern follows Gmail / GitHub / Linear / Vercel: - Leftmost cell on each row is a checkbox; data-id binds to session id. - Header checkbox is tri-state (empty / indeterminate / all-on-page) driven by a single source of truth (selectedSessionIds: Set<string>). - Shift-click on a row checkbox does range-select from the last-clicked anchor to the current row, applying the new checked state to every row in between (macOS Finder / Gmail convention). - Contextual action bar slides in above the table when ≥1 row is selected, showing "N selected · Delete selected · Clear (Esc)". - Confirm modal shows count + first 3 session id prefixes + "and N more" for sanity-check before destructive action. - Esc key clears the selection (no-op when an input is focused). - Selection is keyed on session id and reapplied after each diff render — innerHTML rewrites no longer clobber checked state. - Vanished sessions auto-drop from selection. Backend: - PruneOptions gains sessionIds?: string[] alongside the existing single sessionId. Unknown ids are silently skipped (race-safe with concurrent prunes). Composes with sessionId if both are set. - handlePrune plumbs sessionIds through; the bulk-delete action calls /api/sessions/prune once with all ids. - 3 new selectSessionsToRemove tests: bulk select, unknown-id skip, sessionId+sessionIds composition. Tests: 266 → 269 (+3). Typecheck + build clean. |
||
|
teamchong
|
0a25aeb51d |
feat(dashboard): honest saved_pct uncertainty band (saved_pct_low..saved_pct_high)
The headline used to publish a precise saved_pct (e.g. "68.2%") even
when α was thin / fallback. Switch to a range whenever uncertainty is
wide enough to matter — point estimate only when the spread is tight.
- fitCosts() returns alpha_low / alpha_high (and chars_per_token bounds)
from p10/p90 of per-sample residual α = (tokens - β·pixels) / chars
across the fit-sample ring. For n≤3 uses min/max; for larger n uses
10th/90th percentile. Order-clamped against the central point so a
noisy sample can't put low > point or high < point.
- baselineCost() gains an optional alphaOverride knob that varies ONLY
the chars-per-token side while keeping the image-cost regime stable
(β·pixels when fit is present, 2500/img fallback otherwise). Without
this knob, an earlier draft mixed regimes and produced
saved_pct_low > saved_pct — silently incoherent.
- Totals tracks three parallel baselines: effectiveInputBaselineEst
(point), …Low, …High. Every event updates all three.
- /proxy-stats exposes saved_pct, saved_pct_low, saved_pct_high plus
the matching effective_input_baseline_est_{low,high} and
saved_effective_tokens_{low,high}. Bounds clamped ≥ 0.
- HTML headline collapses to point estimate when (high − low) < 5 pp;
otherwise renders "lo–hi%" as the headline with point as subtitle.
- When fit is null (n<3), wide fallback brackets (0.15 / 0.25 / 0.50)
produce a deliberately wide range so the headline shows a calibrating
state instead of a fake-precise point estimate.
- 5 new tests: alpha_low ≤ alpha ≤ alpha_high invariant, chars/tok
bound inversion, serveStats bound monotonicity, null-fit range shape,
and warmup-baseline ordering.
Tests: 261 → 266 (+5). Typecheck + build clean.
|
||
|
teamchong
|
17b77c7492 |
fix(transform): count tools[], tool_use, thinking blocks in outgoingTextChars
The walker was missing three big text-bearing paths the upstream tokenizer sees: top-level `tools[]` (name + description + JSON schema), `tool_use` blocks (name + serialized input), and `thinking` blocks. Under-counting these inflates α in `tokens ≈ α·textChars + β·pixels`, which biases the dashboard's `saved_pct` HIGH — the honesty cost HANDOFF item #1 called out. - Walker now sums: system + tools[].{name,description,input_schema} + per-block text/tool_use.input/tool_result.{tool_use_id,content}/ thinking. JSON.stringify via safeStringifyLen() for object fields so the count matches what the API actually serializes. - Lift the walker call out of the compression-success path: also run it on the below_min_chars and not_profitable early returns. Without this, α was sampled only from "big enough to compress" requests. - New `outgoingTextChars walker (denominator honesty)` describe block in render.test.ts: 7 tests covering baseline, tools[], tool_use, tool_result (string + array forms with image skip), thinking, unknown-block skipping, and a JSON.stringify(req).length upper bound asserting walker ≥ 50% of envelope on a content-dense request. - Confirms walker stays strictly below the JSON envelope (structural keys/braces still excluded — that's intentional). Tests: 254 → 261 (+7). Typecheck + build clean. |
||
|
teamchong
|
269ae4503a |
dashboard: drop constrained CV gate to 0.1% + surface text_cv on badge
Live diagnosis: real steady-state single-session traffic has 1-2% text
CV turn-to-turn (verified 2026-05-19: six consecutive turns at 132,738
/ 133,021 / 133,610 / 134,105 / 134,718 / 138,097 chars = 1.33% CV).
The previous 2% gate I'd set was still too tight — the constrained fit
never activated, dashboard stayed on stale-constants regime indefinitely.
The CV gate was over-protective. With β pinned, α = Σ(r·x)/Σ(x²) is
mathematically well-defined and numerically stable at any cvX > 0. The
gate only matters for splitting two collinear parameters (joint mode).
For constrained mode I'm replacing the gate with a confidence display:
- MIN_CV_CONSTRAINED = 0.001 (0.1% — purely defensive against
byte-identical sample collapse)
- text_cv + pixels_cv added to cost_fit JSON response
- badge surfaces 'text CV X.X%' so operator sees confidence directly
- constrained-mode badge color tracks CV:
blue = robust (CV ≥ 5%)
amber = thin but measured (1% ≤ CV < 5%)
gray = mean-ratio estimator (CV < 1%)
This is the right shape: don't gate, show the number. The operator can
judge whether '38% saved at text CV 1.3%' is trustworthy enough.
Tests:
+ new: 6-sample 1.33% CV fixture activates constrained mode + reports
cv fields
+ replaced: 'below 2% floor rejects' → 'byte-identical (cv=0) rejects'
unchanged: joint mode + collinear-pixels constrained mode
|
||
|
teamchong
|
91c969b593 |
dashboard: split CV threshold — 5% joint / 2% constrained
Live-traffic diagnosis: typical single-session warm-cache load has the same cached image across turns (pixel CV = 0%) AND text variance of 3-4% (verified 2026-05-19: 117k/127k/128k chars across three turns = 3.84% CV). Both columns failed the original 5%-everywhere gate, so the constrained fallback I just shipped never actually activated — the dashboard fell straight through to wandering stale constants. The 5% gate exists to stop joint OLS from splitting two indistinguishable parameters. In constrained mode β is pinned (no split to make), so we're fitting one parameter through the origin — 1-D OLS is numerically stable at much lower variance. 2% is conservative; even 1% would converge. New thresholds: MIN_CV_JOINT = 0.05 (unchanged — both columns must vary) MIN_CV_CONSTRAINED = 0.02 (NEW — text only, β-pinned through origin) Real-world impact: live single-session traffic now activates constrained mode after 3 compressed turns instead of waiting for cross-session variance that may never arrive. Dashboard label flips from amber 'stale constants' to blue 'constrained · α-only' — operator sees a measured α within ~30 seconds of starting a fresh proxy. Tests: + new: live-traffic-shape sample activates constrained mode (3.84% CV) + new: below-2% floor still rejects (no garbage 1-D fits) unchanged: 2-D collinear cases still null, joint-fit at high CV still returns mode='joint' |
||
|
teamchong
|
9d3984fd03 |
render: multicol gutter divider (light-gray boundary cue for OCR)
Paint a 1-pixel-wide mid-gray (191/255) vertical line in the center of
each gutter at multicol >= 2. Same 'visible whitespace' principle as the
U+2192 tab arrow — surfaces structure that pure whitespace alone leaves
ambiguous, at near-zero pixel-token cost (DEFLATE collapses identical
mid-gray runs to ~5 bytes per gutter).
Why now: the user noted gutters relied on whitespace alone, which is
exactly the OCR-reordering risk HANDOFF flagged for multi-col 2 (lines
322-326). An explicit boundary cue lets us push wider multicol or wider
cols in future without spooking the vision encoder into reading
left-to-right across columns.
Tests:
- byte-determinism (existing test still passes)
- inflate IDAT + assert mid-gray pixels at expected divider X
- numCols=1 path stays byte-identical to renderTextToPngs (deterministic
cache_control story for single-col deployments)
Cost: ink=64 pre-invert → 191 post-invert. Doesn't compete with glyph
ink (which lands at 0). At β=0.001776, the divider's ~1568 pixels cost
~3 tokens total per multi-col image. Trivial vs the OCR-clarity win.
|
||
|
teamchong
|
3893fa6726 |
dashboard: constrained-fit fallback (β=1/750) + regime label on HTML
When the live OLS can't identify β separately (pixels column collinear because a single cached image dominates the 50-sample ring), fall back to α-only fit with β pinned to Anthropic's published rate. Returns mode='joint' | 'constrained' so the HTML can tell the operator which regime the headline saved_pct came from. Three-state badge under the reduction card: - green 'empirical · joint OLS (n≥10)' = both rates measured, confident - amber 'empirical · joint OLS (n<10, tentative)' = both measured, thin - blue 'constrained · α-only (β pinned 1/750)' = α measured, β=Anthropic - amber 'stale constants (no fit yet)' = pre-fit, treat skeptically Closes the 'wandering ±15pp until the fit fires' gap from HANDOFF: even under single-session traffic the dashboard now shows a measured α rather than oscillating between joint-OLS-degenerate and stale-constants. |
||
|
teamchong
|
36ce17ebb6 |
dashboard: collinearity guard — refuse to activate fit when α/β unidentifiable
# The user-visible bug
Showed wandering saved_pct from oscillating OLS:
n=4: α=0.509, β=0.00387, saved_pct=20.1%
n=8: α=0.879, β=0.00104, saved_pct=25.0% (same session, 4 more samples)
# The root cause
A single Claude Code session sends warm hits with the SAME cached image
on every turn → `pixels` column has ZERO variance across samples. The
regression `tokens ≈ α·text + β·pixels` only identifies α and β
SEPARATELY when both columns vary. With pixels constant the model
degenerates to `tokens ≈ α·text + c` where c = β·p̄ — OLS distributes
the per-sample cost between α and β however the residual noise pushes,
and the split flips wildly as samples accumulate.
The det != 0 numerical check missed this because text has tiny non-zero
variance, so the matrix isn't EXACTLY singular — just near-singular,
which is enough for the headline to wander by ±15 pp.
# The fix
Add a coefficient-of-variation guard. Both columns must have CV (stdev
÷ mean) > 5% before fitCosts() returns non-null. Below that, the fit
is statistically meaningless and we fall back to the stale-constants
baseline. Better to show a stable known-wrong number than an unstable
unidentifiable one.
This is the OPPOSITE direction from the earlier 'cache_read===0' gate
relaxation (which was the right call — warm hits DO contribute to the
sample ring), but at a different layer:
- Gate at sample ingestion: liberal (any compressed request with
full usage triple)
- Gate at fit ACTIVATION: strict (require non-collinear samples)
# Known remaining biases (HANDOFF documents)
1. `outgoing_text_chars` undercounts — doesn't include `req.tools`
text. Measured slope α≈1.1 across collinear-pixels samples implies
~0.9 chars/token, vs Opus 4.7 reality 2.7-3.6. Probably ~half the
tokenized chars are missing from our walker.
2. Even with #1 fixed, β identification needs cross-session variance
(samples with different cached images). Either wait for cache TTL
expiry, drive multi-session traffic, or add a constrained-fit
fallback that pins β = 1/750 (Anthropic's published rate) and
solves α only.
# Tests
tests/dashboard-fit.test.ts: replaced the warm-hit-with-constant-pixels
test (which passed only because the old code had no CV guard) with two
new tests:
- returns null when pixels column is constant (single-session case)
- returns null when text_chars is constant (mirror case)
- activates the fit when both columns vary > 5% (cross-session-style
samples reconstruct α≈0.286, β≈1.5e-3 to within ~10%)
248 tests pass (up from 246).
# Milestone status
The dashboard is now HONEST (refuses to claim numbers it can't
measure) but NOT yet CALIBRATED (still on stale constants until cross-
session samples land). HANDOFF.md captures the path to calibration.
|
||
|
teamchong
|
f14a159530 |
dashboard: drop the bogus 'cache_read===0' fit-sample gate
The earlier gate (`cache_read === 0 && cache_create > 1000`) was wrong.
Anthropic's tokenizer is deterministic on the request bytes — cache state
changes BILLING (cache_read is discounted), not token count. The total
tokenization of the request body is always:
total_tokens = input + cache_create + cache_read
…regardless of how it was split across cached/uncached portions. So warm
hits carry the same body-token information as cold misses; the regression
`tokens ≈ α·text_chars + β·pixels` is solvable from either.
The old gate locked the fit out of all normal traffic: warm hits ARE the
steady state once a Claude Code session has run a few turns. The user
correctly noticed that cost_fit stayed null forever even with the proxy
processing real requests, because every request after the first hits the
prompt cache.
# The fix
- Remove the `cache_read === 0` requirement.
- LHS becomes `input + cache_create + cache_read` (was `input + cache_create`).
- Keep the 1000-token floor (filters trivial no-system requests where
tokenizer overhead dominates).
- Rename `coldMisses` / `ColdMissSample` / `COLD_MISS_CAP` → `fitSamples`
/ `FitSample` / `FIT_SAMPLE_CAP` since the samples are no longer cold-
miss-gated.
# Collinearity guard already handles the single-session case
The OLS no-intercept solver uses det = sxx·syy − sxy². With pixels held
constant at p̄ (same cached image across a session's warm hits) but
text_chars varying turn-to-turn, det = p̄²·(n·Σx² − (Σx)²) which is
positive by Cauchy-Schwarz whenever text varies. So a single Claude Code
session can populate the fit on its own — no need to spawn N sessions.
# Tests
New tests/dashboard-fit.test.ts (5 cases) locks in:
- null below 3 samples
- warm cache hits DO seed the ring (the regression we're guarding)
- LHS = input + cache_create + cache_read (cold vs warm with identical
body lands on the same regression line)
- 1000-token floor still filters trivial no-system traffic
- compressed=false passthrough requests don't contaminate the fit
246 tests pass (up from 241), typecheck + build clean.
|
||
|
teamchong
|
937f0f8817 |
prune: drop dead 'placement' + 'compressSystem' knobs (system-field branch was API-rejected)
The `placement === 'system'` branch attached image blocks to the `system`
field, which Anthropic 400s with `system.N.type: Input should be 'text'`.
The existing CLI help text already warned about this. No tests exercised
the dead branch (grep for `placement.*'system'` finds zero call sites).
`compressSystem` was only consulted inside that same dead branch, so it
went out with it. The other granular `compress{Tools,Schemas,Reminders,
ToolResults}` flags + the `compress` master switch stay — they're useful
debugging toggles with active call sites.
Touched:
- transform.ts: drop both fields from TransformOptions + DEFAULTS;
collapse the if/else around system-rewriting into a bare block (no
re-indentation noise, no behavior change).
- node.ts: drop --placement CLI flag, PLACEMENT env, help text entries,
startup log column, and the TransformOptions wire-up.
- worker.ts: drop PLACEMENT from Env + transform options.
- render.test.ts: refresh three comments that referenced the now-gone
"default placement is 'user'" language; the test bodies were already
correct and didn't reference the removed knobs.
20-line net reduction, 241 tests pass, typecheck + build clean.
|
||
|
teamchong
|
a52423870b |
dashboard: wire empirical cost fit into headline savings math + handoff
`DashboardState.fitCosts()` runs OLS over a rolling 50-event ring of
cold-miss (image_pixels, outgoing_text_chars, cache_create_tokens)
samples. `baselineCost()` and `estImageTokens()` consult the fit on
every event:
- `cost_fit != null` (N ≥ 3 samples) → headline saved_pct switches to
the live empirical rates: txtReplaced = compressedChars × α,
imgTokens = imagePixels × β.
- `cost_fit == null` → falls back to the stale constants
(OPUS_IMAGE_TOKEN_COST = 2500, CHARS_PER_TOKEN = 4) — same behavior
as before, so nothing breaks while data accumulates.
# Why a rolling ring, not seeded from history
Old events may carry the new instrumented fields but come from a
different renderer config (cell size, multiCol, atlas profile) or a
different model (Opus 4.6 vs 4.7 tokenizer). Mixing them in gives a
misleading α/β that lags reality. Each restart starts the fit empty
and refills from live traffic, so flipping `multiCol`, swapping
fonts, or upgrading the model auto-flushes stale samples.
# Endpoint
`/proxy-stats` JSON payload now carries:
"cost_fit": {
"alpha": 0.3, "beta": 0.0053,
"chars_per_token": 3.3, "pixels_per_token": 189,
"single_col_tokens_per_img": 4180,
"multicol2_tokens_per_img": 8500,
"n": 12
}
null until ≥3 cold-miss samples land. Guards against degenerate fits
(collinear data, negative rates) by returning null rather than feeding
the dashboard noise.
# Math
Solves `tokens ≈ α · text_chars + β · pixels` via 2×2 normal
equations on the centered design matrix. Rounds α to 4 sig figs, keeps
β at 6 sig figs (β is tiny ~5e-3 so it needs the resolution).
# Open
Once N stabilizes at ~10+, re-grade the hardcoded
`OPUS_IMAGE_TOKEN_COST` and `CHARS_PER_TOKEN` constants to match the
empirical numbers. That just makes the fall-back path agree with
reality, doesn't change live behavior. HANDOFF.md captures the
verification recipe and priority queue for the next session.
|
||
|
teamchong
|
f1688e9f46 |
transform+render: R2 multi-col default-on + per-request pixel/char instrumentation
Two pieces of work landing together because they're interleaved in transform.ts and need to ship at the same time for the dashboard fit-wiring in the next commit to work. # R2 multi-column rendering (default `multiCol=2`) Single-col packing wastes ~68% of Anthropic's 1568×1568 image budget on real Claude Code tool docs (~38 chars/row). The row-aware break- even gate correctly rejects compression on those slabs — proxy ends up doing nothing on the heaviest requests. Multi-col packs N text columns side-by-side per image so each image covers `N×LINES_PER_IMAGE` wrapped lines, dropping image count by ~N and crossing break-even. - render.ts: new `renderTextToPngsMultiCol(text, cols, numCols)` with column-major layout (col 0 fills top-to-bottom, then col 1, …). Plus `multiColWidth` / `maxFittingCols` helpers. CLI auto-clamps numCols so the canvas can never exceed the 1568 px width cap. - transform.ts: `multiCol` option threaded through; gate, image-count estimator, paging budget, and all render paths take `numCols`. Intro text now describes column reading order when numCols>1 so the OCR pass doesn't read row-wise across columns. - Host wiring: node.ts (--multi-col / MULTI_COL=N, default 2), worker.ts (MULTI_COL env, default 2). Startup log shows multi_col=N. - numCols=1 stays a byte-identical passthrough to renderTextToPngs so flipping back can't regress the cache_control hit rate. # Empirical-cost instrumentation The dashboard's `OPUS_IMAGE_TOKEN_COST = 2500` and `CHARS_PER_TOKEN = 4` are stale pre-Opus-4.7 constants. Fact-checked: 4.7 changed the TEXT tokenizer only (~1.20× inflation on English, ~1.36× on TypeScript); image cost is unchanged for same-size images. Direction of error in the dashboard headline: it UNDERSTATES text savings. Ditched the "guess from one cold-miss" approach in favor of real instrumentation that pins both axes at once: - TransformInfo.imagePixels — Σ width×height across every image this request (static slab + reminders + tool_results + history collapse). Threaded through textToImageBlocks, collapseHistory. - TransformInfo.outgoingTextChars — every char of TEXT remaining in the transformed outgoing body, computed in one walk before serialization (countOutgoingTextChars). - Tracker event schema picks up image_pixels + outgoing_text_chars. - history.ts: HistoryCollapseInfo.collapsedImagePixels accumulates. The regression `tokens ≈ α·outgoing_text_chars + β·image_pixels` over N cold-miss events solves for empirical chars/token (α) and pixels/token (β) under the live model — no assumed constants. # Tests - 8 new render tests covering the multi-col passthrough contract (numCols=1 byte-identical to renderTextToPngs), canvas dimensions, image-count halving, determinism, charsRendered sum, estimator parity, CLI clamp, and CJK wide-glyph wrap. - paging.test.ts "counts multiple tool_results" threshold loosened from >770k to >700k to cover both single-col and multi-col budgets at the same maxImagesPerToolResult cap. - All 241 tests pass, typecheck + build clean. |
||
|
teamchong
|
aed76dbdc2 |
transform: compact JSON in tool docs (kills the sparse-fill crisis)
renderToolDoc pretty-printed every input_schema with 2-space indent,
putting each property on its own line. At cols=100 this wasted ~70% of
horizontal space — schemas rendered to many short rows instead of a few
dense ones.
Live measurement on 2026-05-19: 150 KB of pretty-printed tool docs in a
177 KB combined slab → 31 static-slab images per request at 40% fill.
With the new row-aware break-even gate (
|
||
|
teamchong
|
f0176c1432 |
transform: row-aware break-even gate stops net-loss compressions
isCompressionProfitable previously estimated image count as
ceil(chars / charsPerImage), which assumes uniform full-width line-fill.
renderTextToPngs actually budgets by wrapped visual rows (141 per image),
so newline-heavy code/logs and short-line content render to many more
images than the chars-based estimate predicts.
Live impact: dashboard reported -69% reduction (net loss). 33 compressions
averaged 5494 chars/image (39% fill ratio) — gate under-counted cost by
~9× on sparse content and let net-losing compressions through.
Fix:
- isCompressionProfitable now accepts the full text string; uses
estimateImageCount (the same row-aware math renderTextToPngs uses).
- Adds an optional imageCountCap for paths that truncate before rendering
(tool_results). Cost bounded by cap; savings still measured against
the full pre-truncation length, so 500KB logs still profit.
- All 4 production call sites in transform.ts pass the string. The
tool_result sites also pass maxImagesPerToolResult as the cap.
- Number-arg form retained for back-compat with the existing
isCompressionProfitable(N) unit tests.
Tests:
- 3 new unit tests pin the row-aware regression: sparse content rejected,
dense content accepted, capped content (paging) accepted.
- 7 integration tests previously used 'claude.md\n'.repeat(5000) — sparse
fixtures the new gate correctly rejects. Updated to dense fixtures
('x'.repeat(40000) or single-line repeats without embedded newlines)
so they still exercise the cache_control / determinism / reminder /
tool_result code paths they're meant to.
|
||
|
teamchong
|
8fbd53ea3e |
dashboard: fix bogus savings math + emit compressed_chars
The dashboard headline was showing 0% saved because the math was
apples-to-oranges:
- orig_chars = static slab + tool docs ONLY (~135 KB)
- image_count = ALL images including reminders + tool_results (~33)
- txtReplaced = 135k/4 = 33.7k tokens
- imgTokensEst = 33 × 2500 = 82.5k tokens
- extraText = max(0, 33.7k - 82.5k) = 0 ← clamped negative
- baseline == paid → 'saved 0%' headline despite real savings happening
Two fixes:
1. Track 'compressedChars' separately from 'origChars'. It accumulates
only when a block actually image-renders (passes the break-even gate
AND any per-block min thresholds). Includes the static slab + all
compressed reminders + all compressed tool_results.
2. Remove the Math.max(0,...) clamp in baselineCost(). Negative savings
surface honestly — they're the signal that small-block compressions
are cost-bleeding, which the break-even gate is supposed to prevent.
Wires:
- TransformInfo.compressedChars (new field, mandatory).
- All 4 image-render success paths accumulate (static slab + reminder
+ tool_result string branch + tool_result array branch).
- TrackEvent.compressed_chars (new optional field, emitted when > 0).
- dashboard.baselineCost() takes compressedChars now; clamp removed.
- sessions.aggregateSessions() also uses compressed_chars + honest math.
229/229 tests passing. Bundle stays at 496 KB gzipped.
Post-restart, the dashboard 'tokens saved' will show real numbers
including negatives. Empirically expect ~1-5% net positive in
steady-state cache-hit traffic.
|
||
|
teamchong
|
4f5b10aaee |
host: wire Variant C history-image opts through node + worker CLI
Threads compressHistory / historyKeepTail / historyMinPrefix through CliOpts, env vars (COMPRESS_HISTORY, HISTORY_KEEP_TAIL, HISTORY_MIN_PREFIX), and CLI flags (--history, --no-history, --history-keep-tail N, --history-min-prefix N). Same wiring on Workers host. Startup log reports history=on/off. TrackEvent extended with 4 new fields (collapsed_turns, collapsed_chars, collapsed_images, history_reason) — emitted only when non-zero/defined. Defaults: compressHistory=false (opt-in given round-3 MARGINAL rating). 229/229 tests green, 496 KB gzipped. |
||
|
teamchong
|
97fdee8421 |
history: Variant C closed-tool-sequence collapse + honest per-session savings
Two distinct improvements in one slice:
1. History-image compression (Variant C — opt-in, default OFF).
New src/core/history.ts module:
- findClosedPrefixBoundary(): walks tool_use_id openSet, returns largest
fully-closed prefix index. Mid-flight tool_use blocks bail the collapse.
- blocksToText(): linearises content blocks for the history image. Drops
thinking blocks (per Opus 4.7+ docs: only most-recent must be bit-perfect),
renders tool_use args + tool_result inner text, emits [image] placeholder
for nested images.
- collapseHistory(): orchestrates. Applies isCompressionProfitable() gate.
Returns a synthesized {role:'user', content:[text,image,text]} prepended
message + the elided turns dropped from messages[].
New TransformOptions: compressHistory (default false), historyKeepTail (4),
historyMinPrefix (10). Wired through src/node.ts + src/worker.ts as
COMPRESS_HISTORY / HISTORY_KEEP_TAIL / HISTORY_MIN_PREFIX env vars.
Cache_control: history-image carries NO cache_control on this first cut.
Static slab keeps its 1 breakpoint. Conservative — telemetry will reveal
whether the cache-miss rate on history images warrants the round-3
rebalance in a follow-up slice.
29 new tests in tests/history.test.ts: closed-prefix detection (parallel
tool calls, mid-flight, openSet straddling), block linearisation, full
collapse flow, isCompressionProfitable gate firing on small histories.
2. Per-session savings: honest formula (fixes task #63).
Previous src/sessions.ts computed saved = orig_chars - image_bytes — a
nonsense comparison of chars to PNG bytes, then clamped via Math.max(0,...).
Result: charsSaved/tokensSavedEst was 0 for every session including ones
with hundreds of MB of compressed traffic.
New formula mirrors src/dashboard.ts:
textTokens = orig_chars / 4
imageTokens = image_count × 2500
tokensSaved = textTokens − imageTokens (CAN be negative)
Negatives are NOT clamped — they surface the cost-bleed on small-block
compressions that the per-block break-even check is supposed to prevent.
2 new tests: positive-net case, all-negative case (must report negative).
229/229 tests passing. Bundle ~500 KB gzipped.
|
||
|
teamchong
|
04b1526bd5 |
transform: large-tool_result paging + adaptive break-even derivation
Two related improvements:
1. Per-tool_result paging / truncation. When a single tool_result would
render to more than maxImagesPerToolResult images (default 10), the
source text is truncated head+marker+tail BEFORE rendering. Classifier
picks the truncation strategy:
- structured (JSON/YAML/diff): head-only, tail is repeating elements
- log (timestamps / [LEVEL] prefixes): 60% head + 40% tail
- other (prose, dumps): same head/tail split
The marker tells the model what was elided so it can reason about the
gap: '[ pixelpipe paging: omitted N lines (K chars) ... ]'. Keeps
requests under Anthropic's 100-image-per-request cap even when a single
tool dumps a multi-megabyte log.
2. Adaptive CHARS_PER_IMAGE. Previously hardcoded at 14_100 (Unifont 10px
@ cell 5×11). Now derived at module load from ATLAS_CELL_H + render.ts
constants. When gen-atlas regenerates with a different font/size, the
break-even threshold automatically updates: at Cozette 4×7 the chars/
image jumps from 14.1k → 22.2k and the break-even drops from ~10k → ~3k.
Single source of truth in src/core/.
Wires:
- TransformOptions.maxImagesPerToolResult (default 10)
- TransformInfo.truncatedToolResults + omittedChars
- TrackEvent.truncated_tool_results + omitted_chars (emit when > 0)
- Two new exported helpers: estimateImageCount, classifyContent, truncateForBudget
- Paging fires before image-encoding at both tool_result branches
(string content and array-of-text-blocks content)
Tests: +28 paging-specific (paging.test.ts), 199/199 total green.
Bundle delta: negligible (~3 KB raw, ~1 KB gzipped). 488 KB dist gzipped.
Closes the paging-implementer worktree merge that was deferred earlier.
|
||
|
teamchong
|
b5d768fcdc |
dashboard: sessions UI + stats panel + smooth updates + cleanup
Pivot from prior CLI subcommands (stats, sessions) to dashboard-based UI. bin/cli.js becomes a minimal proxy starter; all functionality lives at http://127.0.0.1:47821/. New endpoints: GET /api/sessions.json — session list + Claude Code mapping GET /api/sessions/:id.json — session detail (bodies redacted by default) POST /api/sessions/prune — dry-run-first cleanup with atomic rewrite GET /api/disk.json — events.jsonl + 4xx-bodies disk usage GET /api/stats.json — full-history aggregate (replaces CLI 'stats') GET /sessions/:id — HTML detail page Dashboard HTML extended with sessions panel + stats panel + cleanup panel. Smooth-poll: 2s tick for legacy live counters, 5s tick for sessions/stats/disk. Session rows diff-rendered (no full-table flash). Destructive actions show window.confirm() with dry-run preview of impact. Session mapping: synthetic ID from first_user_sha8 (already on every event) + opportunistic enrichment by fingerprinting against ~/.claude/projects/*.jsonl. Falls back gracefully if Claude Code's session dir is missing. Atomic events.jsonl rewrite (write to .tmp, fsync, rename) preserved from the prior CLI design — trigger is now HTTP POST. +51 new tests (33 sessions + 18 dashboard-api). 149/149 total. |
||
|
teamchong
|
e7489ca1fb |
transform: per-block compression break-even check + honest dashboard
URGENT production-cost fix. Empirical data (orig_chars=169779, image_count=88)
showed we were paying ~220k image tokens to replace ~42k text tokens — a
net LOSS of ~178k tokens per request. Dashboard hid this via Math.max(0,...).
Two changes ship together:
1. isCompressionProfitable(textLen) gate in transform.ts. Before any
per-block image encode (static slab, reminder, tool_result), compute:
estImages × TOKENS_PER_IMAGE vs textLen / CHARS_PER_TOKEN
Compress only when the image side wins. Constants:
CHARS_PER_IMAGE = 14100 (Unifont 10px, cell 5×11, 100 cols)
CHARS_PER_TOKEN = 4
TOKENS_PER_IMAGE = 2500 (empirical, history-researcher round 3)
Result: blocks under ~10k chars stay as text. Static slab still
compresses (always above break-even).
2. Coarse thresholds bumped as fast-path:
minReminderChars 1000 → 5000
minToolResultChars 5000 → 10000
(Cover the obvious-no cases without invoking the precise check.)
3. Dashboard honesty: removed Math.max(0,...) clamp in baselineCost().
Negative savings now display as negative (no more hidden cost-bleed).
4. Telemetry: passthrough_reasons.below_threshold + .not_profitable
counters surface how often each gate fires.
After restart, dashboard 'saved' should jump from clamped-0 to a real
positive number (~1-3% on aggregate; static slab compression is the
only block consistently profitable at our current cell).
|
||
|
teamchong
|
e328c93d52 |
atlas: bit-pack pixel storage (1 bit/pixel) for 3.15× bundle shrink
Unifont glyphs are inherently 1-bit (every pixel is 0 or 255). Storing as
8-bit bytes wasted 7 bits per pixel.
Changes:
- scripts/gen-atlas.ts: emit ATLAS_PIXELS as bit-packed MSB-first. OFFSETS
now indexes bits not bytes. Per-cell paint thresholds R-channel at 128
defensively (Unifont is already 0/255).
- src/core/render.ts:blitGlyph: byte read replaced with bit extraction
(~3 extra ops per pixel; negligible). max() blending removed since bit
data is binary, no AA edges, no glyph overlap.
- src/core/atlas.ts: regenerated with the new format. Header comment
documents the bit-extraction formula.
- tests/render.test.ts: +1 structural assertion that ATLAS_PIXELS.byteLength
matches the bit-packed total.
Bundle size:
atlas.ts raw: 5.47 MB → 1.06 MB (5.2× shrink)
atlas.ts gzip: 1.55 MB → 479 KB (3.2× shrink)
dist gzipped: 1.57 MB → 488 KB (3.15× shrink, well under Workers
free-tier 1 MB cap)
Pre-deflate shrink is 8× (theoretical max). The 3.15× post-deflate
result reflects that deflate was already exploiting long runs of 0/255
in the 8-bit data; the bit-packed bytes are denser and less compressible.
Render output is byte-identical to pre-bit-pack (storage format change
only). 108/108 vitest, 4/4 shell tests, typecheck + build clean.
|
||
|
teamchong
|
efcd2a5e3e |
render: R1 whitespace minify before wrap
Adds minifyForRender() as step 1 of wrapLines() pipeline. Strips trailing whitespace per line and collapses 4+ consecutive newlines to 3 (= 2 blank lines). Leading whitespace + mid-line spaces preserved (structural). HANDOFF item R1 predicted ~1.5-2× chars per image. Empirical measurement on real corpora (v3 corpus, all .md docs, all .ts source, 2.3 MB live conversation jsonl) showed 0% reduction — modern editors strip trailing whitespace on save by default. R1 is shipped as defensive infrastructure that fires when log-dump-heavy / unminified content arrives. +9 tests covering trailing-strip, blank-line collapse, structural-space preservation, and combined-with-tab-expansion pipeline. 107/107 vitest. |
||
|
teamchong
|
cc7f0ed4b8 |
tests: align tab-expansion test expectations with the visible-arrow shape
Previous test expectations had been written against an older silent-space tab expansion. The arrow form (\t → "→" + padding to tab stop) was already in src/core/render.ts; this just brings the tests into agreement. Includes 7 expandTabsInLine tests covering col 0/1/2/3 tab boundaries, the fast-path for tab-free lines, and EAW-aware width counting for tabs after CJK glyphs. Total: 98/98 vitest passing. |
||
|
teamchong
|
f199640bdc |
ship Unifont @ 10px multilingual rendering + 400 diagnosis fixes
- Switch font: JetBrains Mono @ 15px → Unifont 16.0.04 @ 10px. Sparse atlas with double-width EAW dispatch for CJK. Full BMP profile (35,501 glyphs) covers Latin/Cyrillic/Greek/Hebrew/Arabic/CJK/Hangul plus Letterlike, Misc Technical, Block Elements, Geometric Shapes, Misc Symbols, Dingbats, Enclosed Alphanumerics, all box-drawing. - Fix cell-height bug (was hardcoded 9; now probed from font metrics via Math.ceil(maxAscent+maxDescent)). Previously clipped the top 2 rows off every CJK glyph since the original Unifont ship. - Expand tabs to spaces with 4-column tab stops, EAW-aware. Was the silent-drop source for 99.6% of dropped chars in production. - Add dropped_codepoints_top telemetry (top 20 per event, only emitted when drops > 0) so future coverage gaps surface in events.jsonl. - Raise minToolResultChars 2000→5000 and minReminderChars 1000→2000 to stop net-losing tokens on small compressions, per ~2,500 tok/img empirical measurement (N=33 cold-miss events). - Fix dashboard per-image-cost formula: was pngBytes/375 (~190 tok), now imageCount * 2500 (the measured empirical rate). Reduces the inflated "tokens saved" headline to honest numbers. - Schema-stub fix in compressSchemas path: preserveSchemaShell() retains type / properties / required / enum / oneOf / anyOf / $ref / numeric constraints recursively (depth 20), strips only descriptions and metadata. Prevents 400 errors on first-tool-call requests. - Capture full gzipped 4xx body + per-event req_body_sha8 for upstream error diagnosis. Sidecar to ~/.pixelpipe/4xx-bodies/ when over 32 KB inline cap. Workers-safe via CompressionStream + Web Crypto. - scripts/restart.sh: pnpm-aware one-command kill+rebuild+start with graceful SIGTERM→SIGKILL escalation, build-failure abort, port-in-use precheck, --no-build flag for fast iteration. Shell-tested. - Bundle 1.58 MB gzipped on Node. 91/91 vitest + 4/4 shell tests. |
||
|
teamchong
|
241cdbb6aa |
build: pin pnpm@10.21.0 as the package manager
- Add `packageManager` field so corepack uses the pinned version - Add `preinstall: npx only-allow pnpm` guard to block accidental `npm install` (npm exits non-zero with a clear redirect) - Add `engines.pnpm: >=10` requirement - Bundles the previously-uncommitted restart + test:restart script entries (already in use via the diagnostic-instrumentation slice) |
||
|
teamchong
|
b6f2b65b9a |
dashboard: correct the docstring's renderer-width number
Per the renderer-analyst's audit of src/core/render.ts, our renderer's fixed width is `2*PAD_X + DEFAULT_COLS*ATLAS_CELL_W = 8 + 100*9 = 908px`, NOT the 1466 the previous docstring claimed (which mistook Anthropic's max image width for ours). True worst-case is 908×1568/750 ≈ 1898 tokens, a typical image at this workload is ~170 tokens (well under what the byte-derived estimator computes already). No formula change — `estImageTokens(pngBytes) = pngBytes/375` was always size-agnostic. This is a comment-only fix so a future reader doesn't repeat the mistake. |
||
|
teamchong
|
b49764e64c |
docs: add TRANSFORM_INFO.md (compression strategy reference)
A single canonical document explaining how pixelpipe compresses Claude Code requests end-to-end. Covers, in order: 1. Why this proxy exists 2. The static / dynamic split (DYNAMIC_BLOCK_TAGS, KNOWN_STATIC_TAGS) 3. What the final request looks like (placement='user' layout) 4. The cache_control budget (the one invariant that matters: ttl='1h') 5. Per-message compressions (reminders + tool_results, no cache_control) 6. Determinism and fingerprints (systemSha8, claudeMdSha8, firstUserSha8) 7. The unknown-tag canary 8. The savings math (effectiveCost / baselineCost from src/dashboard.ts) 9. What deliberately did NOT get built 10. Wiring map Intended as the file a new contributor reads after CLAUDE.md but before touching src/core/transform.ts. HANDOFF.md is session-state and rotates; README.md is install/run; CLAUDE.md is rules; this is the architecture reference that explains WHY the code looks the way it does. Authored by the doc-writer teammate, KNOWN_STATIC_TAGS section post-edited to reflect that |
||
|
teamchong
|
be372a0c3a |
legacy: drop python reference oracle (live savings confirmed at 98.7%)
HANDOFF item #4 was "Delete legacy/python/ — but only after #1 confirms live savings." Item #1 (live cache-rate validation) now passes: over 232 compressed POST /v1/messages calls in a 46-minute real-traffic window, cache hit rate by tokens is 98.7%, clearing the HANDOFF 60% bar by 38 points. The Python reference has done its job. Removes (1831 deletions): legacy/python/proxy.py (975) full Python proxy implementation legacy/python/diff_transform.py (213) body-diff parity oracle legacy/python/cli.js (170) shim legacy/python/visual_compare.py (150) PNG-diff parity oracle legacy/python/diff_render.py (136) render-only parity oracle legacy/python/gen_atlas.py ( 75) atlas generator (TS port owns this now via scripts/gen-atlas.ts) legacy/python/install.js ( 50) installer shim legacy/README.md ( 35) Comment / doc updates to remove references: README.md drops the "## Legacy" section CLAUDE.md drops the "## Parity oracle" section + tree entry src/core/transform.ts header comment no longer cites legacy/python src/dashboard.ts header comment notes the formulas were ported, not where from docs/TRANSFORM_INFO.md edits land in the separate TRANSFORM_INFO commit The byte-output determinism the Python oracle existed to guard against is now pinned by tests/render.test.ts ("renders identical input to byte- identical output (determinism = cacheability)") and the systemSha8 fingerprint in every JSONL row. If a future change ever breaks determinism, the test will catch it before merge, and the production-deployed cache hit rate will catch it within minutes after. Gates: typecheck clean, 38 tests pass, build clean dist/node.js. |
||
|
teamchong
|
96e3422250 |
dashboard: derive image-token estimate from actual PNG bytes, not worst-case constant
The dashboard's `baselineCost` was using a hardcoded `IMAGE_TOKEN_ESTIMATE
= 3066` per image (the worst-case 1466×1568 image at Anthropic's
~ceil(W·H/750) rate). Our renderer almost never produces full-size
images — typical workload renders ~1466×90 px per image — so the constant
overestimates by roughly 15×. Concrete effect: with `image_count = 90`
and `orig_chars = 173835`, `imgTokensEst = 275940` overwhelms
`txtReplaced = 43459`, the `extraText = max(0, …)` clamp collapses to 0,
and the dashboard reported `saved_pct = 0%` on a session that was
actually getting 98.7% cache hit rate by tokens (savings-analyst
findings against the user's real events file).
Replace the constant with a per-image estimate derived from actual PNG
byte count:
raw_pixels ≈ png_bytes × 2 (grayscale zlib compresses dense glyph
rows to ~50% of raw)
token_est ≈ raw_pixels / 750 (Anthropic's published rate)
= png_bytes / 375
Helper `estImageTokens(pngBytes)` encodes this. `baselineCost` now takes
`imageBytes` (which TransformInfo has always carried) instead of
`imageCount`. The `expected_image_tokens` RecentRow field — shown in the
dashboard "img tok" column — is updated to the same estimator so live
and replayed rows agree. The legacy `IMAGE_TOKEN_ESTIMATE` constant is
removed; nothing else referenced it.
No test changes — existing tests don't exercise saved_pct math (the
dashboard module isn't in the test surface) and the formula change is
purely an estimator refinement. Verified against the live events file:
saved_pct now shows positive numbers reflecting the actual cache wins.
Open follow-up: the renderer is producing many short images instead of
fewer tall ones (~30 wire-bytes per source-text-char), and that's a
separate render.ts investigation. This commit doesn't change that — it
just makes the dashboard math honest about the current renderer output.
|
||
|
teamchong
|
e219b37cd5 |
transform: KNOWN_STATIC_TAGS allowlist (silence <types> canary)
The canary in splitStaticDynamic flags tag-shaped blocks in the static slab that aren't in DYNAMIC_BLOCK_TAGS — designed to catch a new Claude Code release that ships a per-turn tag we'd accidentally bake into the cached image. In production it has been firing on `<types>` for 93% of real requests (125 of 134 in the user's events log), which is noise: `<types>` is part of Claude Code's built-in tool documentation, not a per-turn rotation. Add a second list, KNOWN_STATIC_TAGS, that mirrors DYNAMIC_BLOCK_TAGS in shape. The unknown-tag computation now subtracts BOTH lists, so a tag is only "unknown" when it's in neither the per-turn set nor the known-static set. Seed KNOWN_STATIC_TAGS with ['types']. Updates the warn log in src/node.ts to mention both options so a reviewer of a NEW unknown tag knows which list to extend. Adds a regression test pinning that <types> in the static slab no longer produces info.unknownStaticTags. |
||
|
teamchong
|
a5f61ba6ee |
transform: compress large tool_result text content across user messages
Tool results (Bash output, Read file contents, etc.) accumulate as the
session grows. Once produced they're static — same bytes turn after turn
— but the model gets billed for them on every call until they fall out
of cache. Image-rendering them at the source compounds the savings
across the whole conversation.
Walks ALL user messages (not just the first — that's the reminder-
compression scope) and rewrites tool_result blocks whose content is at
or above minToolResultChars (default 2000, matching legacy/python/proxy.py):
- content: string → replace with image block(s) (whole content goes)
- content: array → walk inner blocks, replace each TextBlock ≥ threshold
with image blocks; leave ImageBlocks and short text
alone
- is_error: true → leave the whole block alone. Anthropic rejects
images inside is_error tool_results
(`tool_result with is_error=true cannot contain images`).
Like reminder compression these images carry NO cache_control — the
system+tools image is our only breakpoint. The savings are 65-73% of
input tokens per compressed block on Opus 4.7.
Wire-up:
- src/core/transform.ts: TransformOptions.compressToolResults (default
true) + minToolResultChars (default 2000). TransformInfo.toolResultImgs
counter. Reuses textToImageBlocks / approxBlockBytes from commit 3.
- src/core/tracker.ts: TrackEvent.tool_result_imgs flows from the
transform info.
- src/worker.ts: COMPRESS_TOOL_RESULTS + MIN_TOOL_RESULT_CHARS env vars.
- src/node.ts: CliOpts fields, parseCli cases (--no-tool-results,
--min-tool-result-chars), help text, env-var summary, `tr+N` in the
per-request console line when tool_result_imgs > 0, and tool_results=
in the startup config summary.
Tests:
- "compresses large tool_result text content across user messages"
confirms info.toolResultImgs ≥ 1, the tool_result block's content
becomes an array of image blocks, and those images carry no
cache_control.
- "leaves is_error tool_results untouched (Anthropic forbids images
there)" confirms info.toolResultImgs is 0 and the content stays a
plain string when is_error: true.
|
||
|
teamchong
|
40037568e2 |
transform: compress <system-reminder> blocks in the first user message
Claude Code re-injects the same long <system-reminder> blocks (CLAUDE.md
hints, task-tools reminders, the dynamic skills list, etc.) into the
FIRST user message on every turn. They're text, so they're not part of
the system+tools image we already cache — they hit the wire fresh every
turn. This rewrite renders any reminder block ≥ minReminderChars (1000
by default, matching legacy/python/proxy.py) to PNG images placed inside
the user message content array, just like the static slab.
Why: the system+tools image is the only cache_control breakpoint we're
allowed (Anthropic caps at 4 and Claude Code uses the others). Per-block
reminder images therefore carry NO cache_control — they're per-turn
content that the model OCRs without prefix-cache benefit. The win is
token cost, not cache: reminders are 65-73% cheaper as PNG than as text,
and a typical turn has 1-3 of them.
Layout inside the first user message after compression:
[intro text] ← static (helps OCR framing)
[image block(s)] ← static; LAST has cache_control
↑ cache breakpoint (the only one)
[End of rendered context.] ← static text closer for the image
[processed existing content] ← per-turn (incl. reminder images)
Wire-up:
- src/core/transform.ts: new TransformOptions.compressReminders (default
true) + minReminderChars (default 1000). Two new helpers,
textToImageBlocks() (wraps renderTextToPngs + makeImageBlock with
no cache_control) and approxBlockBytes() (b64 → byte-count for the
imageBytes telemetry, no second base64 round-trip). The placement
='user' branch now walks `existing` and rewrites long reminders.
- src/core/tracker.ts: TrackEvent.reminder_imgs flows from
TransformInfo.reminderImgs through toTrackEvent so `pixelpipe stats`
and the dashboard can attribute compression contributions.
- src/worker.ts: COMPRESS_REMINDERS + MIN_REMINDER_CHARS env vars.
- src/node.ts: matching CliOpts, parseCli cases (--no-reminders,
--min-reminder-chars), help text, env-var summary, and `rem+N` in
the per-request console line when reminder_imgs > 0.
Tests:
- "compresses long <system-reminder> blocks in the first user message"
confirms info.reminderImgs ≥ 1, the reminder text is gone from
user-content text blocks, the user's actual prompt survives, and the
new reminder image blocks carry NO cache_control.
- "leaves short <system-reminder> blocks alone (below minReminderChars)"
confirms info.reminderImgs is 0 and the reminder text passes through
as text.
|
||
|
teamchong
|
ba13bf3a42 |
dashboard: live HTML + JSON endpoints in the Node host
Ports the live dashboard from legacy/python/proxy.py into TypeScript.
Node-only by design — the Workers host doesn't expose a dashboard (use
Workers Logs / Logpush for that).
New module src/dashboard.ts:
- DashboardState keeps a 50-entry ring buffer of recent requests, plus
running totals for the savings-vs-baseline computation (same formula
as the Python: input + cache_create*1.25 + cache_read*0.10). One
latest PNG is held in memory for the preview pane.
- Replay-on-startup: reads the tail of ~/.pixelpipe/events.jsonl so the
UI doesn't show an empty table after a process restart. Cumulative
totals are NOT replayed (file may have rotated; double-counting beats
starting from zero badly here? — no, starting fresh is the lesser
evil after rotation, hence "best-effort replay of the ring buffer
only").
- Four HTTP handlers: serveHtml (the inline dashboard page),
serveStats (JSON aggregate), serveRecent (JSON ring buffer),
servePng (the latest preview image). dashboardPath() classifies a
path so the Node server can intercept before the proxy handler.
- Inline HTML uses CSS object-position to crop the preview at the
image's native resolution — no server-side PNG decode needed.
Plumbing changes:
- src/core/transform.ts: capture info.firstImagePng / firstImageWidth /
firstImageHeight after rendering. NOT persisted to JSONL; toTrackEvent
drops them. Memory cost is bounded (only the latest image survives,
the dashboard replaces on every update).
- src/node.ts: import DashboardState, instantiate, seed via replay,
call dashboard.update(e) inside onRequest BEFORE tracker.emit (the
tracker strips firstImagePng). The HTTP handler intercepts the four
dashboard paths and answers from DashboardState; everything else
falls through to the proxy.
- src/node.ts / src/worker.ts: bump the CLI/env default for PLACEMENT
from 'system' to 'user' so the runtime path matches transform's
default (
|
||
|
teamchong
|
e0950efa6f |
transform: default to placement='user' and stamp ttl='1h' on image cache_control
Two protocol-correctness fixes that have to ship together. 1. **Default placement: 'system' → 'user'.** Anthropic's `system` field is typed as `string | TextBlock[]` — image blocks there come back as `400 system.N.type: Input should be 'text'`. Images must go into a user-message content array. 2. **ttl='1h' on the image's cache_control.** Anthropic enforces the rule "ttl='1h' must not come after ttl='5m'" in processing order (tools → system → messages). Claude Code marks its own user-message content with ttl='1h'; if we leave ttl unset it defaults to '5m', our block lands first in processing order, and the request 400s the moment Claude Code's own 1h breakpoint shows up later in the conversation. The existing image-placement tests now look in messages[0].content instead of out.system, and a new test pins the ttl='1h' invariant directly. |
||
|
teamchong
|
3109a3b5a4 |
cli: add 'pixelpipe stats' subcommand for offline log analysis
Reads ~/.pixelpipe/events.jsonl (or --file / PIXELPIPE_LOG) and prints
a one-screen summary of how the proxy is doing:
- request counts by status bucket (2xx/4xx/5xx)
- compression rate (compressed vs passthrough)
- latency p50/p95/p99 (duration + first-byte separately)
- aggregate Anthropic token usage (input/output/cache_create/cache_read)
- cache hit rate computed two ways (by tokens, by events)
- top skip reasons (so we see WHY a request wasn't compressed)
- top cwds with per-project compression ratio
- system_sha8 distribution → 'reuse rate' = how often the cached
payload is being re-sent. High reuse + low cache_read means
something is breaking cache_control.
- unknown_static_tags warning section if Claude Code shipped a new
dynamic tag that we're silently baking into the image.
--json mode dumps the raw Summary as JSON for downstream pipelines.
Streaming reader so a 100 MB log file doesn't OOM. Pure aggregator
(fold/newSummary/renderTextReport) is unit-tested; only the file IO
shell is untested.
Dispatched from src/node.ts BEFORE parseCli so its flags don't collide
with the proxy's flags. The CLI shim in bin/cli.js stays unchanged.
Tests: +6 (status bucketing, compressed/passthrough split, token agg,
cwd & sha histogram, unknown-tag collection, text report rendering).
|
||
|
teamchong
|
b9c438b275 |
host: wire FileTracker (Node) + JsonLogTracker (Worker) into per-request flow
Node host now appends one JSONL event per request to ~/.pixelpipe/events.jsonl (override with --events-file or PIXELPIPE_LOG). - Size-based rotation at 100 MB (old file → events.jsonl.1). - Sync writes — overhead is ~1ms, dwarfed by upstream latency. - Failures degrade silently: a broken disk disables the tracker but never breaks a request. - Flushed and fsync'd on SIGINT/SIGTERM so the last batch isn't lost. - Opt out via --no-track or PIXELPIPE_TRACK=0. Worker host uses JsonLogTracker with console.log; Cloudflare ingests this as Workers Logs, and Logpush can ship it to R2/S3 as JSONL — same shape Node writes locally, so analysis tooling works on either. Console log line on both hosts now also includes tokens/cache_read inline (e.g. 'compressed 4799ch → 2img/12343B tokens=1234+56 cache_read=800') so cache effectiveness is visible at a glance even without parsing the JSONL. Unknown-tag canary fires a stderr warning when info.unknownStaticTags is non-empty — surfaces Claude Code releases that introduce new dynamic tags before they tank the cache hit rate. Help text and HANDOFF will be updated in the next commit; for now the new flag drops the need for --exclude-dynamic-system-prompt-sections. Tests: +5 (toTrackEvent flattening, JsonLogTracker sink behavior, error swallowing, minimal-event handling, noop). |
||
|
teamchong
|
102a0966f5 |
transform: add determinism test + unknown-tag canary
Two cheap guardrails based on architectural review: 1. Determinism test: renders the same input twice and asserts byte equality of the produced PNGs. Without this guarantee, identical system prompts on consecutive turns would yield different image bytes, silently destroying the cache hit rate that justifies the whole proxy. The test currently passes — locking it in so a future refactor can't regress it without noticing. 2. Unknown-tag canary: splitStaticDynamic now sniffs the *static* slab for any <tag>...</tag>-shaped block whose name isn't in DYNAMIC_BLOCK_TAGS. Surfaced via info.unknownStaticTags and emitted on the tracker event as unknown_static_tags. If Claude Code ships a new per-turn tag in a future release, it'll show up in logs within hours instead of being silently baked into the cached image (where it would tank cache hit rate with no error). Also adds tracker.ts (Tracker interface, TrackEvent shape, JsonLogTracker, noopTracker, toTrackEvent normalizer). Hosts will wire these in the next commit. |
||
|
teamchong
|
cb8377a6e4 |
proxy: tee upstream body and extract Anthropic usage tokens
ProxyEvent gains two new fields: - usage: Anthropic's input/output/cache_creation/cache_read tokens. - firstByteMs: ms to upstream response headers (durationMs now measures end-to-end since the event fires after usage is extracted). Implementation (teeForUsage): - Tee the response body. Client gets one side immediately; we read the other in the background. - SSE: scan up to 64 KiB for 'event: message_start' + the data: payload. Usage is always in the first event so this terminates within ms of upstream first byte. - JSON: buffer up to 4 MiB and parse. - Error responses (status >= 400) and bodyless responses skip extraction. - Drains the tee fully in the background to avoid backpressure. Without this, the proxy had no way to validate its own value proposition. Now we can compute cache hit rate, real input-token savings, etc. directly from log events. Tests: +3 (JSON usage, SSE message_start usage, error skips extraction). |
||
|
teamchong
|
99559c463b |
transform: add sha8 fingerprints (system/claude_md/first_user)
Three short hashes carried in TransformInfo: - systemSha8: sha256[0..8] of the actual image-bound text. Stable across turns proves cache_control is doing its job. If it drifts unexpectedly, that's a cache leak to investigate. - claudeMdSha8: hash of just the CLAUDE.md slab (heuristic header detection). Lets us bucket requests by project even when cwd is absent or anonymized. - firstUserSha8: hash of first user message (4 KiB cap). Rough thread id since the wire protocol has none. All async via Web Crypto so Node + Workers share one impl. Hashing is in parallel via Promise.all; ~0.1ms overhead. +2 tests (systemSha8 invariant under dynamic-block change, firstUserSha8 format). |
||
|
teamchong
|
364373618f |
transform: extract env fields (cwd, branch, platform, today) into info.env
New EnvFields type and extractEnvFields() helper. Parses the <env> block Claude Code injects, plus 'On branch' / 'Branch:' lines that appear in <git_status> or context blocks. Read-only — telemetry signal only, never mutates the request. Per-project compression ratios, repeat-CWD detection, today's-date drift checks all become possible from logs now. +2 tests (full env block extraction, no-env-no-info). |
||
|
teamchong
|
84b4a3a59a |
transform: split static/dynamic system text for cache stability
Claude Code injects per-turn blocks (<env>, <context>, <git_status>,
<directoryStructure>, <system-reminder>) into the system prompt.
Previously those got rendered into the image alongside the static
slab (CLAUDE.md, agent defs, tool docs), which kills Anthropic's
prompt cache: image bytes drift turn-to-turn → 0% cache hits.
Now we split:
- static (cacheable) → image with cache_control: ephemeral
- dynamic (per-turn) → plain text block AFTER the image
- billing line → also AFTER the image (was BEFORE, also a
cache-killer — bug fix as a side effect)
Net effect: stable cache key on the big slab, model still sees cwd /
branch / today's date. Removes the need for users to pass
--exclude-dynamic-system-prompt-sections to claude.
TransformInfo now carries staticChars / dynamicChars / dynamicBlockCount
for downstream telemetry.
Tests: 8 existing pass unchanged + 3 new (env survives as text after
image, cache_control on image only, all-dynamic input is pass-through).
|
||
|
teamchong
|
6b50f94e0b |
Rewrite README, CLAUDE.md, HANDOFF for TypeScript implementation
README: Node + Workers quickstart, config table, architecture diagram,
limitations, dev commands. Legacy Python pointer.
CLAUDE.md: agent orientation — file layout, constraints (no node: in core,
no new runtime deps, don't hand-edit atlas.ts), verification
recipe, Anthropic API constraints we can't regress.
HANDOFF.md: current status (TS port lands, parity not yet verified),
what works, what doesn't, key design decisions, prioritized
list of follow-ups (parity test, live savings test, Workers
deploy, delete legacy/).
|
||
|
teamchong
|
f91c149d05 |
Port to TypeScript with dual Node + Cloudflare Workers targets
Runtime is pure Web Standard APIs (fetch, Request, Response, Uint8Array,
CompressionStream, crypto.subtle, btoa) — zero runtime dependencies.
Core (src/core/, runs identically on Node 18+ and Workers):
- atlas.ts auto-generated 9x15 JBM glyph atlas, base64-inlined (17KB)
- png.ts minimal grayscale PNG encoder via CompressionStream
- render.ts text → packed PNG, soft-wraps at 100 cols, ≤1568px tall
- transform.ts request body rewriter (system + tool docs → image blocks)
- proxy.ts fetch-handler that transforms then forwards to Anthropic
- types.ts Anthropic Messages API types we touch
Adapters:
- src/node.ts node:http server + CLI flag parsing
- src/worker.ts export default { fetch } for wrangler dev/deploy
Build tooling:
- scripts/gen-atlas.ts @napi-rs/canvas → atlas.ts (build-time only)
- scripts/build.mjs esbuild Node bundle; wrangler handles Worker
- tsconfig.json strict, ES2022, Workers types
Tests (vitest): 8 passing — PNG signature, base64 round-trip, single +
multi-image renders, transform no-op + compress paths, billing-line strip,
tool fold + stub.
E2E smoke: 16K char system → 2 PNGs (36KB) → mock upstream, 34ms.
Next: replace docs, then verify byte-output parity against legacy/python.
|
||
|
teamchong
|
437afa4dfe |
Restructure: archive Python+Rust, prep for TypeScript rewrite
- Move Python proxy + scripts to legacy/python/ (parity oracle during port) - Delete Rust port (src/rust/) - history preserved in commits |
||
|
teamchong
|
f20207cd7b |
Rename: claude-image-proxy -> pixelpipe (Tier 1: code + docs)
Project rename for the Show HN launch with framing:
'pixelpipe — what if Opus saw a UI instead of just append-only logs?'
Renamed everywhere:
- kebab-case 'claude-image-proxy' -> 'pixelpipe' (42 occurrences)
- snake_case 'claude_image_proxy' -> 'pixelpipe' (5 occurrences)
- 14 files touched: package.json, Cargo.toml, Cargo.lock, *.md,
bin/cli.js, scripts/install.js, src/proxy.py, src/rust/src/main.rs,
src/rust/examples/{render_sample,transform_only,diff_render}.rs,
src/zig/build.zig
Effects on build artifacts:
- npm package name -> 'pixelpipe'
- Rust binary name -> target/release/pixelpipe (was claude-image-proxy)
- Python cache dir -> ~/.cache/pixelpipe/ (was ~/.cache/claude-image-proxy/)
NOT YET DONE in this commit (Tier 2+3, handled in next commit):
- Repo on-disk directory still ~/Downloads/repos/claude-image-proxy/
- Cache dir on disk still ~/.cache/claude-image-proxy/ — Python proxy
currently writes here. Will be migrated alongside the running services
in a coordinated stop/move/restart sequence.
Validation: scripts/diff_transform.py still produces exactly 1 structural
diff (the PNG bytes, expected — different fonts Python/Menlo vs
Rust/JetBrains Mono). No regression from the rename.
|
||
|
teamchong
|
f094649831 |
Rust transform: match Python's body output byte-for-byte (except PNG)
Three bug fixes surfaced by scripts/diff_transform.py harness, which
synthesizes a realistic Claude Code request body and runs it through both
Python transform_request() and Rust transform(), then JSON-structurally
diffs the outputs.
Before these fixes, harness found:
- 2 structural diffs in the transformed body JSON
- 4+ TransformInfo divergences
After these fixes, harness finds:
- 1 structural diff: $.messages[0].content[1].source.data
(the base64 PNG, expected to differ — Menlo vs JetBrains Mono fonts)
- TransformInfo still differs on byte-vs-char counts (telemetry-only)
Fixes:
1. src/rust/src/transform.rs:93 — per-tool markdown header
was: '### Tool: {name}\n{desc}'
now: '#### Tool: {name}\n{desc}\n'
Matches proxy.py:230,233. Also restores the trailing newline that
Python emits between tool entries.
2. src/rust/src/transform.rs:123 — tool section markdown header
was: '\n\n# Tool Documentation\n\n'
now: '\n\n#### Tool Documentation\n\n'
Matches proxy.py:248.
3. src/rust/src/transform.rs:265 — preserve system field by default
was: always replaced system with billing_line OR remainder
now: only replaces if billing_line was stripped, otherwise leaves
the original system field untouched.
Matches proxy.py:417 which intentionally leaves system alone in the
default PLACEMENT="user" path 'so Anthropic still has guardrails'.
The image lives in messages[].content as a side-channel; system stays
as original text. This is the behavior our 95+ live Python requests
have validated against Anthropic's API.
4. src/rust/src/lib.rs — expose 'transform' module so harnesses and the
transform_only example can call transform() without going through HTTP.
Added:
- src/rust/examples/transform_only.rs — CLI binary that reads a body
from a file and writes the transformed body + TransformInfo as JSON.
- scripts/diff_transform.py — Python harness that calls both Python
transform_request() and the Rust transform_only binary, then runs a
structural JSON diff with field-by-field TransformInfo comparison.
Validation:
python3 scripts/diff_transform.py
# → 'found 1 structural diff'
# → $.messages[0].content[1].source.data (the PNG, expected)
Remaining known telemetry-only diffs (not blocking the swap):
- system_text_chars: Python counts Unicode chars, Rust counts UTF-8 bytes
- tool_text_added: same UTF-8 issue
- expected_image_tokens: ~30% higher in Rust due to ab_glyph 4px cell
vs PIL 3px cell — Rust images are 33% wider for the same text. Rust
PNG bytes are still smaller overall because of the tight-bound height
(which Python also has now, after the previous baseline commit).
- placement, png_sha8 fields missing from Rust TransformInfo
- min_chars threshold uses bytes (Rust) vs chars (Python)
These can be addressed later for full telemetry parity but the
upstream-visible behavior is byte-equivalent.
|
||
|
teamchong
|
5c2f303cfd |
Fix Python padding bug: single-column images now tight-bound
src/proxy.py:125 was computing image height as tallest = max(lines_per_col, last_col_lines) if c_needed == 1 else lines_per_col The max(lines_per_col, ...) made every single-column render pad to the full MAX_EDGE=1568 canvas height regardless of content, wasting ~99% of the image-token budget for small prompts. Fix: tight-bound the single-column case. tallest = max(1, last_col_lines) if c_needed == 1 else lines_per_col Multi-column (c_needed > 1) is unchanged — all earlier columns must be lines_per_col tall to fit them, so the image must be too. Measured impact (scripts/visual_compare.py): sample before after change ----------- ------------ ------------ ----------- 17 chars 241x1568 241x14 -99% pixels 176 chars 241x1568 241x42 -97% pixels 885 chars 241x1568 241x315 -80% pixels Verified edge cases hold: - empty/short text: still gated by MIN_COMPRESS_CHARS - 5000-char single line: 241x441 (63 wrapped lines, tight) - 80-char boundary: 241x14 (2 lines, tight) - 500-line multi-column: 731x1568 (unchanged, correctly padded) - 20KB realistic system prompt: 68.5% savings (matches existing 65-73%) Added scripts/visual_compare.py to inspect Python vs Rust renderings side-by-side with labels and dimensions. This is the immediate win that's independent of the Rust rewrite — every proxied request gets fewer image tokens starting at the next proxy restart. |
||
|
teamchong
|
482c967270 |
Add Rust port of proxy (unproven, not wired up)
Functional reimplementation in Rust with the same render_chunks/transform
public surface as src/proxy.py. Compiles clean on darwin-arm64, smoke-
tested but NOT YET battle-tested against real Claude Code sessions.
Layout:
src/rust/
Cargo.toml hyper + tokio + reqwest + ab_glyph + png + clap
Cargo.lock committed (application crate, deterministic builds)
src/main.rs clap CLI matching bin/cli.js flags
src/lib.rs module roots
src/font.rs ab_glyph rasterization of bundled JetBrains Mono
src/render.rs newspaper-column text -> PNG (matches Python algo)
src/transform.rs request body transformation (system prompt -> image)
src/proxy.rs hyper HTTP server, upstream forwarding
src/stats.rs persisted savings counters mirroring proxy.py
assets/JetBrainsMono-Regular.ttf OFL-licensed, legally embeddable
examples/render_sample.rs smoke test: render fibonacci snippet
examples/diff_render.rs diff harness companion (paired with
scripts/diff_render.py)
Diff harness:
scripts/diff_render.py feeds same text through Python and Rust
render_chunks, decodes both PNGs, compares
pixel-by-pixel. First run found two divergences:
1. Rust font.rs:43 uses ab_glyph h_advance which returns ~3.4px ceiled
to 4 for Menlo-5pt 'M'; PIL's getlength('M') returns 3.0 exactly.
Every Rust column is 33% wider than Python's.
2. Python proxy.py:125 pads every single-column image to MAX_EDGE=1568
height regardless of content (already noted in baseline commit).
Neither is fixed yet — surfacing the divergences before any swap. Next:
visual inspect both PNGs to confirm Rust output is at least OCR-readable,
then decide whether to fix bug-for-bug compat or correct both sides.
bin/cli.js still spawns Python; this commit only adds the Rust tree, it
does NOT change runtime behavior. The live proxy at :47821 is unaffected.
|