Several families share /v1/responses. Cost, geometry, cache rates, and
whether imaging is even allowed must follow the model that serves the
request. Otherwise Claude and Grok inherit GPT defaults, the gate lies,
and the dashboard reports nonsense.
Claude on Responses:
- Bill images by Anthropic pixel area; cache 0.1x, output 5x.
- Use Anthropic page geometry (312 cols x 728 px). GPT height overstated
image cost ~2.6x and flipped every Opus slab to not_profitable, so
enabled Claude stayed text-only with blank As text / Saved.
History profitability gate:
- Reflow packs hard newlines with an inline ↵ glyph. countVisualRows
still treated ↵ as a row break, so estimateImageCount overstated pages
~6x on reflowed history and collapsed Claude/Grok history never fired.
Match the renderer: only hard newlines start a visual row.
Grok:
- Keep production 5x8 packing and a verbatim fact-sheet for OCR-hard
tokens (paths/hex/ports/camelCase). Measured ~1000 image tok/MPix;
cache 0.25x, output 3x.
- Leave Grok out of DEFAULT_MODEL_BASES (opt-in only, same bar as Opus).
Pure-image exact OCR fails at 5x8; do not image it silently.
Also thread optional per-model render style through the Responses path
and allow eval-only atlas overrides in gen-atlas.
- staleFreshnessHints() rewrites the upstream "(file state is current in
your context - no need to Read it back)" hint into a stale-state warning
at history-serialization time, covering both collapsed same-session
turns and slabs inherited by continuation sessions
- Edit/Write/NotebookEdit stubs gain a live-text same-session Read
precondition (READ_FIRST_TOOLS), riding un-imaged in the tools array
- telemetry: churning_static_tags canary - FNV-1a fingerprint per
session/tag flags slab tags whose content churns and busts the image
cache (TAG_OBSERVATIONS_MAX-capped)
- 5 regression tests in tests/history.test.ts; baseline to beat: 97
read-gate errors across 18 sessions
Volatile env/context text (git status, cwd, date) rode in req.system —
BEFORE the slab anchor in Anthropic's prefix order (tools → system →
messages) — so any git-state change cold-restarted the entire anchored
prefix. Telemetry attribution (events.jsonl 2026-06-26..07-02): 48.8%
of cold-create waste, ~2.6k tokens/session vs ~200 saved by imaging it.
Now appended as a trailing text block on the LAST user message — the
per-turn live tail that re-caches incrementally anyway — placed AFTER
history collapse (never baked into frozen chunks) and after tool_result
compression. Session-stable billingLine/sysRemainder stay in system;
fallback keeps env in system when no user message exists to carry it.
New telemetry field: envRelocatedChars.
Fix#1 (cross-session cache bust): strip the volatile "# Environment"
markdown section (working dir, git status, platform, model ID) out of
the system text BEFORE the static/dynamic split. It carries no XML
wrapper so splitStaticDynamic couldn't catch it, and its git-status
lines change across sessions - baking them into the slab PNG busted
the cross-session cache (system_sha8 717f1fce -> 5efaa4bb for a
one-file edit). Parallel to stripBillingLine: the section re-enters
the system tail as plain text.
Schema handling simplified, knob removed: the Anthropic text-reference
path now renders prose-only tool docs (schemas stay untouched in
tools[] - duplicating them in the reference was pure bloat), while the
GPT imaged path always carries the compact schema (there it IS the
compression). Deletes the {type:'object'} stub + SCHEMA_STRUCTURAL_KEYS
machinery, stripSchemaDescriptions import, and the COMPRESS_SCHEMAS env.
Tests: e2e regressions for anchor relocation onto byte-frozen carry-over
history images and first-collapse anchor staying on the slab; history
relocation fixture grown past the collapseChunk=50 snap (56 messages ->
cutoff 50, frozen windows 11/21/31/41, collapsedTurns 49 - the old
35-message fixture floored to one window and never froze a chunk).
tsc clean; 615/615 tests pass.
The text-based Tool Reference triggered Anthropic's model-cloning
safeguard (stop_reason: refusal -> silent Opus fallback) because the
per-tool stub said docs lived in "the system prompt" — the exact
phrasing the 169521c retrip identified as the trigger for the imaged
slab, now reintroduced in text form.
- Stub now reads: 'ⓘ Full docs: see "## Tool: <name>" in the Tool
Reference section.' — no "system prompt"/"authoritative" framing.
- Tool Reference header is provenance-framed as first-party ("pxpipe
(this user's local proxy) moved the full tool documentation...").
- tracker.ts now maps tool_docs_chars into events.jsonl so refusals
are attributable to the text-reference path.
- Regression test: stub + reference header must never match
/system prompt|authoritative/i and must carry provenance framing
(scoped to the header; quoted third-party tool docs below it may
legitimately contain the phrase).
Verified: 614/614 tests, tsc --noEmit clean, and a live cold-start
through the rebuilt proxy on claude-fable-5 with the text reference
active — no refusal event, no fallback.
The in-image banner announced 'SYSTEM PROMPT + TOOL DOCS ... treat as
authoritative system instructions' inside a user turn. Anthropic's
model-cloning classifier read that as a replayed/extracted prompt and
refused (apiRefusalCategory: reasoning_extraction), forcing Claude Code
to fall back to claude-opus-4-8 — outside compress scope, so the whole
session ran passthrough with zero savings.
Reworded to first-party provenance framing (pxpipe rendered this
session's own configuration; no 'system prompt'/'authoritative').
Two headless probes through :47824 now serve on claude-fable-5[1m]
with the imaged slab (img:4, ~94k chars) and zero refusal events;
identical shape was refused pre-fix (req_011CccRnutLzRJNVZ7bRKG8Q).
Anthropic's safety classifier OCRs rendered images. A *legible* image of the
system prompt + tool docs (bash / permissions / credential / file-write
vocabulary) trips its prompt-injection heuristic and forces a model downgrade
to Opus, which costs far more than imaging the (already prompt-cached) slab
ever saves. Empirically the caption wording is NOT the trigger — the OCR'd
content is — so no caption rewrite fixes it; the slab must stay text.
Gate at the slab profitability check with PXPIPE_TEXT_SLAB=1. Default is
unchanged (legacy imaged slab) so the cache-align / cache-stability suites
stay valid. Both demos now launch the pxpipe proxy with the toggle set.
Verified: identical ~300KB-slab request images 8 PNGs by default, 0 with the
toggle (reason=slab_text). Full suite: 614 passed.
Drop the duplicate CHARS_PER_TOKEN=3.7 from export.ts and import REPORT_CHARS_PER_TOKEN from transform.ts so the reporting-estimate constant has one source of truth. Clarify the dashboard baseline-warmth invariant comment. No behavior change.
The magic-word @pxpipe directive fired unreliably and added per-request body
scanning to the hot path, so drop it entirely: removed the directive module +
its test and unwired it from transform.ts / proxy.ts / node.ts / index.ts.
transformRequest now leaves message bodies byte-identical — no per-request
directive parsing.
In its place, `pxpipe export <paths>` renders code/text to dense PNG pages via
the same SDK primitive the proxy uses, so exported pages are byte-identical to
what ships to the model. Width is locked to proxy density (--cols rejected as
an unknown option, guarded by tests) to prevent drift. Added --open to preview
the rendered pages plus a result footer (artifact summary + token-savings line).
Tests: export.test.ts 64 passed (incl. --open parse + default coverage); full
suite 589 passed / 29 files; typecheck clean.
Intermittent prefix-cache busts on long sessions (#11): the single cache
breakpoint relocated onto the LAST history image — the still-growing chunk
whose bytes change every window advance — so the slab+history prefix
re-created at 1.25x instead of being read.
- relocateAnchorToHistoryImage pins collapseHistory's carry-over ordinal
(the last byte-stable chunk) instead of the last image.
- demoteProtectedHeadText keeps pxpipe's slab scaffolding (images +
fact-sheet + the [End of rendered context.] boundary) verbatim and only
quarantines the user's stale opening turn, so the relocator can still
find the slab anchor (#14 demotion had clobbered the boundary it keys on).
- cachePrefixDigest joins parts with \x00 so block-boundary differences
can't hash-collide into false prefix-stability.
520/520 green, tsc clean.
Read-only digest of the exact pinned prefix (tools+system+messages through
the imaged history/slab boundary, live tail excluded), computed after final
marker placement and emitted per-turn via tracker. Attributes a prompt-cache
bust as pxpipe-side (sha8 drifts turn-over-turn) vs upstream eviction (sha8
stable while cache_create spikes). Behavior-neutral: byte-identical requests,
518/518 green.
Recovers precision-critical strings (paths/ids/versions) from the imaged
slab as cache-stable text so they survive OCR. Byte-identical across turns,
validated on both Claude and gpt-5.5 paths.
Collapsed-history images now carry who-said-what AND when, instead of leaving
the model to reconstruct it from a flattened, role-ambiguous blob (which led it
to resurface the opening turn as if it were the live request).
- Turn-index recency anchor: each collapsed turn serializes as <user t="N"> /
<assistant t="N"> with an ABSOLUTE turn index (larger N = more recent), so the
model can tell turn 1 from turn 60. Absolute (never "N ago"/"i of total") so
frozen chunks stay byte-identical and keep hitting the prompt cache. Mirrored on
the GPT path. (history.ts, openai-history.ts, openai.ts)
- colorByRole: role tags are tinted via a parallel "slot string" carried from
serialize time (structure-through), replacing the parse-back that miscolored a
body quoting a literal <tag>. (render.ts, history.ts)
- neutralizeSentinel: a pre-existing ↵ (U+21B5) in content is swapped to ⏎
(U+23CE) in render-prep so reflow packs newlines instead of bailing to a raw,
unpacked render — common when the content is about pxpipe itself. Render-only;
originals preserved. (render.ts, transform.ts, history.ts, openai-history.ts)
- Per-model GPT vision/render profiles, retunable via PXPIPE_GPT_PROFILES,
behavior-identical to the old hardcoded values. (gpt-model-profiles.ts)
Review fixes folded in:
- Banner intro/outro consolidated to a single source of truth; the GPT path now
aliases the Anthropic constant instead of byte-copying it (can't drift).
- slotCopyBody neutralizes literal slot markers to a width-equivalent control
char (U+0003), not a space the minifier would strip and misalign.
459 tests pass; tsc clean.
Collapse old conversation history into rendered PNG sections so the model
reads a compact image instead of re-billed text, while preserving prompt
caching and tool-call behavior. Measures real vs compressed token/cost.
Core:
- GPT history collapse (openai-history.ts): append-only, o200k token-length
sectioning. Sections seal only at a tool-closed boundary (open call-id set
empty), so a function_call and its function_call_output never split across
the collapse cut. Fixes the OpenAI 400 "No tool call found for function
call output with call_id ..." that hit long Responses-API sessions.
- Anthropic cache contract (history.ts): append-only per-chunk rendering;
cache_control markers are preserved/moved, never added; chunk boundaries
align with caller marker seams for byte-stable prefix caching.
- GPT image budget (openai.ts): detail:'original' for gpt-5.x, flagship
vision-multiplier fix, patch cap; schema-strip preserves real descriptions.
- Savings accounting (openai-savings.ts): cached_tokens + vision-token basis.
Model scope (applicability.ts):
- Default imaged scope = claude-fable-5 + gpt-5.6.
- gpt-5.5 and claude-opus-4-8 stay opt-in: same pipeline, but they degrade
reading dense imaged history (gist drift), so silently imaging them by
default is wrong. Promotion is gated on an OCR/recall threshold.
Dashboard: GPT + Anthropic rendering, per-family model toggles, persisted
metrics, thumbnail-expired session UI, reflow/newline handling.
Tests: cache-alignment (GPT + Anthropic), history sectioning + tool-boundary
invariants, savings, dashboard, sessions/restart-restore. 452 passing.
The history image sits after the slab in prefix order, but the single cache
breakpoint was anchored on the slab. So the largest block — the ~141k-token
history image — only cached when the caller's roaming downstream marker happened
to land after it. When it didn't, the whole history image re-created at the 1.25x
cache_create rate every turn instead of amortizing into 0.1x reads. On a warm
session that turned a real compression win into a net loss (the "411% more"
the dashboard correctly surfaced).
Fix is pure relocation: move pxpipe's single breakpoint from the slab onto the
LAST history image (which sits after the slab), so slab + history cache as one
stable prefix — created once, then read. Marker count is unchanged: pxpipe still
never ADDS a marker (Task #21), it only moves the caller's existing one. When no
slab imaged, the history path keeps its own relocation.
- transform.ts: relocateAnchorToHistoryImage on the collapse-success branch
- history.test.ts: two old "history carries no marker" invariants upgraded to
assert the relocation (last history image carries it; exactly one marker total)
- keepSharp: caller predicate pins exact-match-critical blocks (IDs, hashes,
paths) as text, overriding the chars/token heuristic. Reported via
info.keptSharpBlocks. Never forces imaging; throwing predicate = false.
- emitRecoverable: optional recovery map (block -> original text + provenance)
so a stateful caller can re-inject or re-fetch imaged content. The recover
half of the fidelity contract; documented mitigation for lossy recall.
- edge/Workers-safe: process.env read is typeof-guarded; @napi-rs/canvas moved
to devDependencies (atlas is build-time only, runtime PNG encoder is pure-JS).
- comments: trimmed ~1,640 lines of essay/derivation/changelog narration across
src/ (transform.ts -73%), keeping one-line rationale; deep math points to docs/.
- New exports/types in index.ts + library.ts; +12 tests (keep-sharp, recoverable).
Build clean; 354/354 tests pass.
Fable 5 / Opus 4.8 accept up to 2576px / 4784 visual tokens per image, but a
request with >20 images (pxpipe always sends many) is held to the stricter
<=2000px/side rule, so the real ceiling is ~1932x1932 (1928x1928 = 69x69 = 4761
tokens). MAX_HEIGHT_PX 1568->1932; dense tool/history pages now render at
DENSE_CONTENT_COLS=384 / DENSE_CONTENT_CHARS_PER_IMAGE=92160 (1928x1928 full
page) -- fewer image blocks at the same OCR-validated 5x8 cell. The static slab
is unchanged (313 cols / 1573x1280).
Also fixes a regression the code review caught: the break-even gate,
truncateForBudget, and estimateImageCount predicted against the slab geometry
(313 cols / 159 rows) while the dense renderer emits 384 cols / 240 rows, so
large tool_results were truncated far earlier than the 10-image cap required --
silently dropping output that would have rendered. A per-page char budget is
threaded through the prediction chain (defaults to READABLE so the slab and unit
tests are byte-identical) plus a denseGateGeometry helper, so the gate prices the
same page the renderer actually produces.
New docs/CACHING_AND_SAVINGS.md explains the two things that keep coming up:
1. How pxpipe stays aligned with Anthropic's prompt cache when it images the
static prefix: split static/dynamic blocks first (keep drifting <env> etc.
as cheap text, never imaged), then relocate (never add) the caller's
cache_control onto the last image so the breakpoint anchors at the end of
the stable slab. Covers the cache-friendly splice, the symmetric
anti-flapping burn term, and the horizon-amortized history-collapse gate.
2. How savings are measured without double-counting the cache discount: both
the real request and a count_tokens counterfactual of the original body are
priced with identical cache rates at the same moment, so the discount
cancels in the difference and only the token reduction survives.
Also reconcile the priorWarmTokens JSDoc in transform.ts: the burn term is
applied undivided, matching all three gate call sites and the symmetric
priorWarmImageTokens doc. The '/ N' was stale (the per-turn gate has no
horizon to divide by; the horizon scales the lifetime cost, not the burn).
- ImageEntry carries sourceText (capped 64KiB upstream in transform.ts)
- new GET /api/image-source[?id=N] endpoint
- LatestPng gains a 'view source text' toggle so the operator can see
the exact text -> image conversion side by side
Pixelpipe now always uses the full 1568px canvas (cols=313) and packs up
to 50,000 chars per image. The multi-column code path and the
shrink-to-content path are no longer used by transform.ts - both were
sacrificing token savings on dense content to make sparse content marginally
more readable, and the readability gain wasn't real (the renderer's
output was unreadable in either layout once the content was actually dense).
Key changes:
- READABLE_CHARS_PER_IMAGE: 6_000 -> 50_000
- DEFAULT_COLS: 100 -> 313 (full 1568px / 5px cell)
- shrinkColsToContent() is now a no-op (returns cols unchanged)
- MinToolResultChars / MinReminderChars default to 50k (was 6k)
- transform.ts always renders single-column at full canvas
Practical impact: a 6 KB tool_result that previously rendered as 4 narrow
508x488 pages (1324 image tokens, 86% of text cost) now renders as 1
full-canvas 1568x488 page (~331 image tokens, 22% of text cost) - 4x
more savings on the same content with no quality change.
Rebuilt dist/. Tests: 315/315 pass.
SessionSummary: divide saved_$ by baseline_input_$ (was actual_input + output,
which produced 719% on cheap sessions). Math now matches the user-stated
analogy: baseline=10, actual=5 -> saved=5, 50%.
RecentRequests: stamp full imagePngs[] in TransformInfo so each rendered PNG
gets a ring-buffer id. Row shows a strip of 28px thumbs (one per image)
instead of a single 'view' button, so multi-image requests are visible at
a glance. Thumbs are <button>-wrapped for keyboard accessibility.
The system slab's break-even gate was estimating image cost at the full
configured cols width while the renderer was shrinking the canvas to the
longest wrapped line. Gate and renderer now agree: shrink once up front,
pass the shrunk cols to both evalCompressionProfitability and
renderTextToPngsMulticol.
Two fixes to the image-cost prediction that gates compression:
1. **Content-aware image cost.** The old gate used a fixed
`TOKENS_PER_IMAGE_SINGLE_COL = 2500` constant per image, modelling
every tool_result block as a full 1568px canvas. The renderer
actually produces images sized to the wrapped line count, so a
5,000-char tool_result becomes a ~80px-tall image costing ~54 tokens,
not 2,500. Result: the break-even for small blocks was over-estimated
by ~50x and the gate rejected anything below ~10k chars even though
image-form was cheaper.
Replace `estImages × effectiveTokensPerImage(numCols)` with a single
content-aware `imageTokensCost(text, cols, numCols, imageCountCap)`
that mirrors the renderer's height math:
`tokens = (rows + 1) × overhead + fullImageRows × perRow`, derived
from the cold-miss anchor and applied row-proportionally for partial
final images. Calibrated against N=33 production cold-miss
measurements (median multiplier 1.05, mean 1.32 — high-image case is
~1.05x of the documented `width × height / 750` rate).
2. **Width-shrinking on per-block paths.** For per-block call sites
(tool_result, reminder) the renderer now narrows the canvas width to
the longest wrapped line via `shrinkColsToContent`. Multi-col
layouts already pack tightly; this captures the single-col savings
that were previously paying for whitespace. The gate threads
`shrinkWidth` through to `evalCompressionProfitability` /
`isCompressionProfitable` / `isCompressionProfitableAmortized` so
the prediction matches what the renderer will actually produce.
System-slab and history-collapse paths pass `shrinkWidth=false` —
they fill the configured canvas deliberately.
Knobs removed: `MinToolResultChars` and `MinReminderChars` now
default to 0 (let the math decide). The TransformOptions fields are kept
for env-override observability but no longer set a floor in production.
`isCompressionProfitable` / `evalCompressionProfitability` /
`isCompressionProfitableAmortized` are now string-only at the public
API; the numeric-length fallback is gone (it under-counted images and
silently leaked net-losing compressions through).
Tests: gate-flap and history.test updated for the new physics, but 24
tests in render.test.ts still assert the OLD over-estimation as truth
(e.g. "5,000 chars is not profitable", "7000 chars stays as text").
They need to be deleted or have their expected verdicts inverted — left
for the next session to finish the rewrite cleanly. Typecheck passes.
Measured impact: the production tail's $1.43 flap is gone and per-block
tool_result/reminder content is now genuinely compressed when it would
save tokens, not just when it crosses the (wrong) 10k-char threshold.
Production data (ocproxy claude-opus-4-7, 2026-05-23 09:19–09:21) showed
three-turn sessions still flapping AND paying cache_create on every
turn despite the symmetric burn shipped in 0cfe08a:
09:19 APPLIED cc=156k cr=6.7k $0.99 cold start
09:20 declined cc=181k cr=0 $1.23 FLIPPED to text
09:21 APPLIED cc=156k cr=6.7k $1.01 FLIPPED back to image
The slab gate's gateEval said image was profitable by 154k margin at
the flip turn, but applied=0 — the slab gate had decided to stay in
image mode, then the history-collapse gate (which got hardcoded 0 for
priorWarmTokens AND priorWarmImageTokens) flipped the verdict.
Wire o.priorWarmTokens + o.priorWarmImageTokens through to both
history-collapse call sites so the symmetric anti-flapping burn
applies to every gate that can change the session's bill shape, not
just the slab. Foundational follow-through on the same fix.
Production data (ocproxy Opus 4.7, 2026-05-22 23:15–23:44) showed the
slab break-even gate ping-ponging between text and image modes within a
single session, paying cache_create on whichever side it just flipped
away from. Two flips in 30 minutes cost ~$0.71 + ~$0.40 of avoidable
cache_create on a single conversation.
Root cause: priorWarmTokens was asymmetric. It penalised compressing
when the un-rewritten text path was warm, but did not penalise
decompressing when the rewritten image path was warm. Per-turn cost
favored flipping; the flip forced cache_create on the new side; next
turn flipped back.
This change adds the symmetric counterpart:
text→image flip: warm text invalidated → burn applied to IMAGE side
(existing behavior, priorWarmTokens)
image→text flip: warm image invalidated → burn applied to TEXT side
(NEW, priorWarmImageTokens)
Once a session commits to a mode, staying in that mode is cheaper than
flipping by the burn cost. Mode changes only happen when the per-turn
delta exceeds the burn, which is exactly when the flip is actually
worth the cache_create. Foundational fix — no thresholds.
Observability:
- New exported `evalCompressionProfitability()` decomposes the slab
gate's evaluation into imageTokens, textTokens, burnImageSide,
burnTextSide, and verdict. Same constants and helper as the gate
itself, no drift risk.
- New `TransformInfo.gateEval` field stamps that decomposition into
every applied or not_profitable event so hosts can persist the verdict
margin and prove flap-prevention efficacy from telemetry.
Back-compat:
- New params default to 0; single-arg callers behave identically.
- 315 existing tests unchanged; +7 new tests pin the new semantics
(back-compat, asymmetric ⇒ symmetric, both-sides-warm cancellation,
eval verdict matches gate verdict, burn term math).
The reflow-inimage finding (commit bb9c231) made 5×8 viable, so
DEFAULT_CELL_W_BONUS / DEFAULT_CELL_H_BONUS reverted to 0 — but many
comments and the README still said 7×10 was production.
Touched:
- README.md — eval table, density bullet, sweep narrative,
cell-pitch section, measurement header,
Limitations section
- src/core/render.ts — RenderStyle docstring + cell-bonus docstrings
+ multicol intro comment
- src/core/transform.ts — TransformOptions.reflow docstring, DEFAULTS
comments (minReminderChars floor rationale,
maxImagesPerToolResult cap, reflow gate
rationale, anchor scaling formula, LINES_PER_IMAGE
worked example)
No code behavior changed — every constant still resolves through CELL_W /
CELL_H, and the eval harness still overrides the cell via cellWBonus /
cellHBonus when comparing variants. The 5×8 production path is what was
already shipped on 81550f6 (reflow + grayscale + L1/L2 harness) and
bb9c231 (reflow-inimage).
Pack text into a continuous sentinel-delimited stream (↵ = U+21B5) so
wrapLines fills every row to `cols` instead of leaving dead right-margin.
Adds reflow/dereflow, renderTextToPngsReflow{,MultiCol} variants, a full
test suite, and an A/B eval harness with L1 OCR + L2 session results.
Snap the collapse cutoff to a `collapseChunk`-sized grid (default 50) so
the rendered history image stays byte-identical for a full window of
turns. That lets Anthropic's prompt cache `cache_read` the history
prefix (0.1x) instead of re-billing `cache_create` (1.25x) every turn.
The cutoff floors at `minCollapsePrefix` (clamped to the raw cutoff) so
short conversations still collapse without the boundary drifting.
Also log `history_image_sha8` per request -- a sha8 of the concatenated
history-image base64. An unchanged hash across consecutive collapsed
events is ground-truth proof the prompt cache is being hit; a hash that
moves every turn signals the cache-key drift regression is back.
Production data (ocproxy 7d, Opus 4.7) shows pixelpipe HURTS short
conversations and helps long ones:
input.count <10 : +$0.17 (HURTS) over 14 rows
input.count 10-19: +$0.04 (HURTS) over 22 rows
input.count 20-39: -$0.56 (helps) over 29 rows
input.count 40+ : -$3.21 (helps) over 237 rows
Diagnosis: warm-cache share split.
bucket actual_warm_share baseline_warm_share
<10 60% 100%
10-19 84% 97%
20-39 88% 96%
40+ 92% 58%
For short conversations the un-rewritten path is already nearly fully
warm-cached. Rewriting the prefix bytes (slab compression / image
substitution / history collapse) gives the new prefix a different
cache key, so Anthropic charges cache_create (1.25×) on the new
prefix's first turn — destroying the prior warm cache. The
conversation often ends before the new cache amortizes.
Fix: add a 'priorWarmTokens' option to TransformOptions. When >0, the
break-even gate penalises the image side by
burn = priorWarmTokens × (CACHE_CREATE_RATE − CACHE_READ_RATE)
Per-turn gate eats the full burn; amortized gate spreads textLifetime
across N turns so longer horizons re-accept once savings outweigh burn.
Default 0 preserves byte-identical behavior for callers that don't
plumb it through (cold-start safe).
Tests: 3 new (NaN clamp, per-turn flip, horizon flip). 302 total pass.
N=391 Opus 4.7 measured rows (last 7d) show real chars-per-token = 1.91
(avg_outgoing_text_chars=231,925 / avg_real_input_tokens=115,893). The
older 2.5 constant was calibrated on Opus 4.5/4.6 and over-estimated cpt
by ~30%, under-counting text-token cost and causing the break-even gate
to reject most history-collapse opportunities (283/391 = 72% of measured
ok rows came back history_reason='not_profitable').
Resize one render.test.ts fixture (6060 → 4848 chars) so the 'low cpt
unlocks a previously-rejected slab' assertion still exercises the
intended before/after edge under the new default.
When the cacheable-prefix count_tokens probe failed but the full-body
probe succeeded, the dashboard fell through to `baselineCacheableTokens
?? 0`. That silently treated the unproxied counterfactual as
cold-input-only — charging full $2.50/Mtok input rate on tokens that
demonstrably would have been cache-discounted at 10× cheaper. Net effect:
fabricated "$ saved" headline on every partial-probe row.
The fix:
1. New `TransformInfo.baselineProbeStatus: 'ok' | 'partial' | 'failed'`,
set by the proxy after awaiting both probes. 'ok' means either both
probes resolved OR the original body had no cache_control markers
(cacheable=0 is then exact, not estimated). 'partial' means the
full-body probe resolved but the cacheable-prefix probe didn't.
2. `tracker.ts` persists it as `baseline_probe_status` on the JSONL.
3. `dashboard.ts` gates its savings math on `status === 'ok'` (with a
back-compat path for legacy rows that pre-date the field). Partial /
failed rows now contribute to the request count but not to
baselineInputWeighted / actualInputWeighted / outputWeighted.
ocproxy's host-side dashboard already records baseline_probe_status in
its SQLite schema and is being updated in parallel to honor the same
gating semantic. See ocproxy commit fixing dashboard/src/lib/snapshot.ts
for the matching change.
Steven Chong