ai-space/pxpipe
3
0
Fork 0
mirror of https://github.com/teamchong/pxpipe.git synced 2026-07-22 02:02:51 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix(savings): gate text-counterfactual warmth on static-prefix hash

Browse Source 
A fresh same-session prior within the TTL is no longer sufficient to price
the text counterfactual warm: deriveBaselineWarmth now also requires the
static-prefix hash (system_sha8) to match. When opencode rotates the system
prompt / tool docs mid-session, the cacheable prefix changes, so a text-only
client would hit a new provider cache key too — pricing it warm against a
cold actual fabricated a huge phantom "loss" (the dashboard's 800%-worse
report). cr>0 still rescues a genuine warm read with no in-memory prior.

Wired through update(), replay(), and aggregateSessions via system_sha8.

Also surface losses honestly in the recent table (Saved/lost: negative
deltas in red instead of hidden as "—") and clarify headings as
billing-equivalent input tokens. Docs updated to match.
This commit is contained in:
Steven Chong
2026-06-29 15:59:10 -04:00
parent ec43bf5370
commit ff8a05e30c
9 changed files with 269 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CACHING_AND_SAVINGS.md
+68 -53
View File
@@ -277,24 +277,29 @@ So the baseline is **warmth-aware**, and warmth is the honest **union of two
independent witnesses** that the *text* prefix was cached this turn — warm iff
EITHER fires:
1. **Wall clock** — the same session had a usage-bearing turn **less than
`CACHE_TTL_SEC` (300s) ago**. The text counterfactual is a *separate*
hypothetical client whose prefix is **append-only**, so a recent prior turn
proves that prefix is still cached on Anthropic's side **even when pxpipe
busted its OWN image cache** (`cr = 0`) by re-rendering the prefix in place
this turn. The text's cache fate is **not** tied to the image's: pricing it
warm here correctly *surfaces* the re-imaging loss (cheap warm text < the
imaged actual) instead of hiding it behind a matching cold baseline.
1. **Wall clock + same static prefix** — the same session had a usage-bearing
turn **less than `CACHE_TTL_SEC` (300s) ago**, and the static-prefix hash
(`system_sha8`) matches. The text counterfactual is a *separate* hypothetical
client whose prefix is **append-only**, so a recent matching prior turn proves
that prefix is still cached on Anthropic's side **even when pxpipe busted its
OWN image cache** (`cr = 0`) by re-rendering the prefix in place this turn.
The text's cache fate is **not** tied to the image's: pricing it warm here
correctly *surfaces* the re-imaging loss (cheap warm text < the imaged actual)
instead of hiding it behind a matching cold baseline. If `system_sha8`
changed, this is a different provider cache key, so the prior does not prove
warmth.
2. **Observed read** — `cache_read > 0` directly witnesses Anthropic serving a
cached prefix this turn. This rescues the **first turn after a restart / TTL /
`SESSION_CAP` eviction**, where this process has no in-memory prior yet but the
cache is provably warm.
`cr = 0` does **not** force cold — leg 1 carries cache-busted re-renders within
the window. `cr` is only an *additional* sufficient witness, never a necessary
one. That is the entire difference from the old `cr > 0`-only rule, which treated
image and text as one shared cache slot and so mispriced cache-busted re-renders
**cold**, hiding a real loss (see the audit history below).
`cr = 0` does **not** force cold when the static-prefix hash is unchanged — leg 1
carries cache-busted re-renders within the window. `cr` is only an *additional*
sufficient witness, never a necessary one. A fresh prior with a different
`system_sha8` is cold unless `cr > 0` independently proves a read. That is the
entire difference from the old `cr > 0`-only rule, which treated image and text
as one shared cache slot and so mispriced cache-busted re-renders **cold**, hiding
a real loss (see the audit history below).
```
cacheable = min(baselineCacheable, baseline) // the would-have-cached prefix
@@ -305,8 +310,8 @@ Then price `cacheable` by warmth:
| case | condition | how the text path is billed |
|---|---|---|
| **cold turn** | no fresh prior **and** `cr = 0` (first turn, or a >300s gap that let the entry expire with no observed read) | `cacheable × 1.25 + coldTail × 1.0` |
| **warm turn** | fresh same-session prior <300s ago **OR** `cr > 0` (the union) | `reused × 0.10 + grown × 1.25 + coldTail × 1.0` |
| **cold turn** | no matching fresh prior **and** `cr = 0` (first turn, a `system_sha8` change, or a >300s gap that let the entry expire with no observed read) | `cacheable × 1.25 + coldTail × 1.0` |
| **warm turn** | fresh same-session prior <300s ago with matching `system_sha8` **OR** `cr > 0` (the union) | `reused × 0.10 + grown × 1.25 + coldTail × 1.0` |
where, on a warm turn:
@@ -330,11 +335,11 @@ it never would have paid.
> **Signature note.**
> `computeBaselineInputEff(baseline, baselineCacheable, inputTokens, cc, cr, warm, prevCacheable)`.
> The `warm` flag and `prevCacheable` are produced by **`deriveBaselineWarmth`**
> (the union: `warm = freshPrior(<300s) || cr > 0`) at every call site — including
> the dashboard/sessions replay path, which passes the **persisted** timestamp so
> it reproduces the live decision exactly. Once warm, the magnitude of the read is
> driven by `prevCacheable` (how much prefix carried over), not by the raw `cr`
> count.
> (the union: `warm = freshPrior(<300s, matching system_sha8) || cr > 0`) at every
> call site — including the dashboard/sessions replay path, which passes the
> **persisted** timestamp and `system_sha8` so it reproduces the live decision
> exactly. Once warm, the magnitude of the read is driven by `prevCacheable` (how
> much prefix carried over), not by the raw `cr` count.
Two guard rails short-circuit before the warmth split:
@@ -368,15 +373,18 @@ Two guard rails short-circuit before the warmth split:
> prefix was still cached. Pricing it cold gave the baseline a matching cold
> create, so the row showed ≈0 — **hiding a real re-imaging loss** (the
> inverse of the (2) phantom-saving bug).
> 4. Both of (3) are fixed by the **union**: `warm = freshPrior(<300s) || cr > 0`.
> Leg 1 (wall clock) prices cache-busted re-renders warm so the loss surfaces;
> leg 2 (`cr`) rescues the no-prior post-restart turn. The text counterfactual
> is a *separate* append-only client — its cache fate is decoupled from whether
> pxpipe busted its own image cache.
> 4. Both of (3) are fixed by the **union**: `warm = freshPrior(<300s, matching
> system_sha8) || cr > 0`. Leg 1 (wall clock plus same static-prefix hash)
> prices cache-busted re-renders warm so the loss surfaces; leg 2 (`cr`) rescues
> the no-prior post-restart turn. The text counterfactual is a *separate*
> append-only client — its cache fate is decoupled from whether pxpipe busted
> its own image cache. If `system_sha8` changed, that is not an append-only
> continuation, so the text counterfactual is cold too unless `cr > 0` proves a
> read.
>
> The current model does all of it: split the prefix, gate the read rate on real
> warmth, and take the **union** of the wall-clock prior and the observed read so
> neither a phantom saving (2) nor a hidden loss (3) can return.
> warmth, and take the **union** of the matching-hash wall-clock prior and the
> observed read so neither a phantom saving (2) nor a hidden loss (3) can return.
> `tests/baseline.test.ts` locks `cold(prefix) > warm(prefix)` and the union truth
> table; `tests/dashboard-api.test.ts` and the sessions replay tests lock both the
> cache-busted-re-render-within-TTL case (warm text, cold image, **loss surfaced**)
@@ -426,13 +434,14 @@ reads **3,000 image tokens** at 0.1× where the text arm reads **27,000** — 9
fewer tokens sitting under the same discount. That reduction, not the cache
discount, is what pxpipe is credited with.
**(b) Cold turn (first turn or a >5-min idle).** Same body, but *genuinely* no
warm cache for either path — no fresh same-session prior **and** `cr = 0`. The
imaged request creates its ~3,000-token image prefix cold: `input_tokens = 2000`,
`cc = 3000`, `cr = 0`. Both legs of the union fail, so the text counterfactual is
priced cold too: it would equally have re-created its prefix. (`cr = 0` *alone*
is not the tell — a `cr = 0` turn that sits <300s after a prior is warm via leg 1;
that's case (c).)
**(b) Cold turn (first turn, changed static prefix, or a >5-min idle).** Same
body, but *genuinely* no warm cache for either path — no fresh same-session prior
with matching `system_sha8` **and** `cr = 0`. The imaged request creates its
~3,000-token image prefix cold: `input_tokens = 2000`, `cc = 3000`, `cr = 0`.
Both legs of the union fail, so the text counterfactual is priced cold too: it
would equally have re-created its prefix. (`cr = 0` *alone* is not the tell — a
`cr = 0` turn that sits <300s after a matching prior is warm via leg 1; that's
case (c).)
```
Counterfactual (text, cold):
@@ -453,17 +462,18 @@ create; the image path eats only `3000×1.25`.
> always read the prefix at `28000×0.10 = 2,800` → `baseline_eff = 4,800`, then
> subtracted the real `actual_eff = 5,750` for a **−950 "loss"** on a turn that
> actually saved 31,250. The union prices it cold the *honest* way — both legs
> fail (no fresh prior **and** `cr = 0`), so the text path re-creates its prefix
> at 1.25× exactly as the imaged path does, and the phantom loss is gone. A
> `cr = 0` turn that *does* sit <300s after a prior is the opposite animal: leg 1
> fires, the text is warm, and pricing it cold would hide a loss — that's (c).
> fail (no matching fresh prior **and** `cr = 0`), so the text path re-creates its
> prefix at 1.25× exactly as the imaged path does, and the phantom loss is gone.
> A `cr = 0` turn that *does* sit <300s after a matching prior is the opposite
> animal: leg 1 fires, the text is warm, and pricing it cold would hide a loss —
> that's (c).
**(c) Cache-busted re-render inside the window (warm text, cold image).** The
*identical* actual request to (b) — `input_tokens = 2000`, `cc = 3000`, `cr = 0`
(pxpipe re-rendered the image prefix in place, so its own image cache missed) —
but this turn lands <300s after a prior that cached a 27,000-token prefix
(`prevCacheable = 27000`, grown 1,000). Leg 1 fires, so the text counterfactual is
**warm** even though `cr = 0`:
but this turn lands <300s after a prior with the same `system_sha8` that cached a
27,000-token prefix (`prevCacheable = 27000`, grown 1,000). Leg 1 fires, so the
text counterfactual is **warm** even though `cr = 0`:
```
Counterfactual (text, warm — leg 1, prefix was cached regardless of the image):
@@ -505,7 +515,8 @@ Every row in `~/.pxpipe/events.jsonl` carries both arms of the same request.
`cache_control` marker (omitted/`0` if the body had no markers)
- `first_user_sha8` — the **session key**. Rows sharing this value are the same
conversation; warmth comes from `deriveBaselineWarmth` over consecutive rows
that share it — a fresh prior within the 300s TTL **or** an observed `cr > 0`.
that share it — a fresh prior within the 300s TTL with matching `system_sha8`,
**or** an observed `cr > 0`.
- the billed `input_tokens`, `cache_create_tokens` (← Anthropic's
`cache_creation_input_tokens`), and `cache_read_tokens` (← Anthropic's
`cache_read_input_tokens`) from the real response. (A 1-hour cache tier, if
@@ -514,21 +525,25 @@ Every row in `~/.pxpipe/events.jsonl` carries both arms of the same request.
Because warmth is a **cross-turn** property, replay isn't purely per-row: group
rows by `first_user_sha8`, sort each group by `ts`, then walk each session in
time order. For each row call
`deriveBaselineWarmth(prev, ts, baseline_cacheable_tokens, cache_read_tokens)` —
the exact function the live path uses — where `prev` is the previous in-session
row (or `undefined` for the first). It returns `{ warm, prevCacheable }` via the
**union**: warm iff a fresh same-session prior exists within the 300s TTL **or**
`deriveBaselineWarmth(prev, ts, baseline_cacheable_tokens, cache_read_tokens,
CACHE_TTL_SEC, system_sha8)` — the exact function the live path uses — where
`prev` is the previous in-session row (or `undefined` for the first). It returns
`{ warm, prevCacheable }` via the **union**: warm iff a fresh same-session prior
exists within the 300s TTL with matching `system_sha8` **or**
`cache_read_tokens > 0`. So a post-restart row with no in-session prior but
`cr > 0` still prices warm, and a `cr = 0` re-render <300s after a prior still
prices warm — neither falls through to cold. (`prevCacheable` follows: the prior
row's `baseline_cacheable_tokens` when the fresh-prior leg fired, else this row's
own `cacheable` when warm via `cr` alone, else `0`.) Feed `(baseline_tokens,
`cr > 0` still prices warm, and a `cr = 0` re-render <300s after a matching prior
still prices warm — neither falls through to cold. A `cr = 0` row after a
`system_sha8` change is cold because the text counterfactual has a different
provider cache key too. (`prevCacheable` follows: the prior row's
`baseline_cacheable_tokens` when the fresh-prior leg fired, else this row's own
`cacheable` when warm via `cr` alone, else `0`.) Feed `(baseline_tokens,
baseline_cacheable_tokens, input_tokens, cache_create_tokens, cache_read_tokens,
warm, prevCacheable)` through `computeBaselineInputEff` and the billed triple
through `computeActualInputEff` (both exported from `src/core/baseline.ts`), sum
the per-row differences, convert with the list ratios above, and you've re-derived
the headline. The live dashboard (`DashboardState`) and the JSONL replay both call
these **same functions** with the persisted `ts`, so the views can't drift.
these **same functions** with the persisted `ts` and `system_sha8`, so the views
can't drift.
### Edge cases worth knowing
@@ -566,8 +581,8 @@ cost over an expected reuse horizon. Savings are then measured by pricing
body with the **same** cache rates (create 1.25×, read 0.1×) and the **same
warmth** — the text counterfactual only reads its prefix cheaply on a turn where
the cache genuinely existed for it (a fresh same-session prior within the 300s
TTL, **or** an observed read `cr > 0`), and pays the create otherwise, exactly as
the imaged path does. Because both arms face the
TTL with matching `system_sha8`, **or** an observed read `cr > 0`), and pays the
create otherwise, exactly as the imaged path does. Because both arms face the
same discount under the same warmth, it cancels in the difference and what
remains as "savings" is only the token reduction from turning dense text into
images, never the prompt-caching discount itself.
src/core/baseline.ts
+17 -4
View File
@@ -18,6 +18,9 @@ export interface BaselineWarmthPrev {
ts: number;
/** Cacheable-prefix tokens measured that turn (0 if the probe missed). */
cacheable: number;
/** Hash of the image-bound/static text prefix. If it changes, the text prefix
* was not the same cache entry even inside the TTL. */
prefixSha?: string;
}
/**
@@ -41,9 +44,11 @@ export interface BaselineWarmthPrev {
* yet cr proves warmth). Without it that turn is priced COLD and fabricates
* an inflated "saved" row — the operator's original reported bug.
*
* Crucially, cr === 0 does NOT force cold (leg 1 carries those turns); cr is only
* an ADDITIONAL sufficient witness, never a necessary one. That is the whole
* difference from the old rule. See docs/CACHING_AND_SAVINGS.md.
* Crucially, cr === 0 does NOT force cold when the prefix hash is unchanged
* (leg 1 carries those turns); cr is only an ADDITIONAL sufficient witness,
* never a necessary one. But a fresh prior with a different prefix hash is cold:
* it is a different provider cache key, not an append-only continuation.
* See docs/CACHING_AND_SAVINGS.md.
*
* @param prev this session's previous usage-bearing turn, or undefined.
* @param nowSec wall-clock seconds of the current turn (replay passes the
@@ -52,6 +57,9 @@ export interface BaselineWarmthPrev {
* when warm only via cr, since cr proves a read but not the split).
* @param cr observed cache-read tokens this turn (the leg-2 witness).
* @param ttlSec cache TTL window (defaults to CACHE_TTL_SEC).
* @param prefixSha stable-prefix fingerprint for the text counterfactual. A
* fresh wall-clock prior only proves warmth when this matches
* the prior turn; otherwise the provider would see a new key.
*/
export function deriveBaselineWarmth(
prev: BaselineWarmthPrev | undefined,
@@ -59,10 +67,15 @@ export function deriveBaselineWarmth(
cacheable: number,
cr: number,
ttlSec: number = CACHE_TTL_SEC,
prefixSha?: string,
): { warm: boolean; prevCacheable: number } {
const age = prev !== undefined ? nowSec - prev.ts : Number.POSITIVE_INFINITY;
const samePrefix = prev === undefined
|| prev.prefixSha === undefined
|| prefixSha === undefined
|| prev.prefixSha === prefixSha;
// Leg 1: a fresh same-session prior within the TTL (wall-clock warmth).
const freshPrior = prev !== undefined && age >= 0 && age < ttlSec;
const freshPrior = prev !== undefined && age >= 0 && age < ttlSec && samePrefix;
// Leg 2: an observed read directly witnesses a warm cache. Union of the two.
const warm = freshPrior || cr > 0;
// Fresh prior → credit its real measured prefix as reused (the reused/grown
src/dashboard.ts
+15 -12
View File
@@ -429,7 +429,7 @@ export class DashboardState {
* TTL), which decides whether the text counterfactual reads (0.10×) or
* re-creates (1.25×) its prefix. Reconstructed identically in replay() from
* persisted `ts`, so live and restored numbers agree. Capped with sessions. */
private baselineWarmth: Map<string, { ts: number; cacheable: number }> = new Map();
private baselineWarmth: Map<string, { ts: number; cacheable: number; prefixSha?: string }> = new Map();
/** Anthropic prompt-cache TTL in seconds; a gap beyond this is a cold turn. */
private static readonly CACHE_TTL_SEC = 300;
/** Hard cap on `sessions` Map entries. Keeps memory bounded in
@@ -657,26 +657,26 @@ export class DashboardState {
// turn)? Decides whether the text counterfactual reads or re-creates. Keyed
// by firstUserSha8; live uses the wall clock, replay() the persisted ts.
const sidNow = info?.firstUserSha8;
const prefixShaNow = info?.systemSha8;
const nowSec = Date.now() / 1000;
const warmthPrev =
typeof sidNow === 'string' && sidNow.length > 0
? this.baselineWarmth.get(sidNow)
: undefined;
// Warmth = honest UNION of two witnesses the TEXT prefix was cached: a
// fresh same-session prior within the TTL (wall clock) OR cr>0 (an observed
// read). The text prefix is append-only, so a fresh prior keeps it warm
// even when pxpipe busts its OWN image cache on a cr=0 re-render (the case
// the old cr-alone rule mispriced cold, fabricating savings); and cr>0
// rescues the first post-restart turn that has no in-memory prior yet (the
// case cr-alone got right but wall-clock-alone would misprice cold,
// fabricating the inflated "saved" row). Centralised in deriveBaselineWarmth
// so update()/replay()/sessions can't drift apart. See docs.
// fresh same-session prior within the TTL AND the same static-prefix hash,
// OR cr>0 (an observed read). The hash guard matters when opencode changes
// system/tool guidance mid-session: same wall clock, different cache key.
// cr>0 still rescues the first post-restart turn that has no in-memory
// prior yet. Centralised in deriveBaselineWarmth so update()/replay()/
// sessions can't drift apart. See docs.
const { warm, prevCacheable } = deriveBaselineWarmth(
warmthPrev,
nowSec,
cacheable,
cr,
DashboardState.CACHE_TTL_SEC,
prefixShaNow,
);
baselineInputEff = creditSaving
? computeBaselineInputEff(
@@ -696,6 +696,7 @@ export class DashboardState {
this.baselineWarmth.set(sidNow, {
ts: nowSec,
cacheable: cacheable > 0 ? cacheable : (warmthPrev?.cacheable ?? 0),
prefixSha: prefixShaNow ?? warmthPrev?.prefixSha,
});
if (this.baselineWarmth.size > DashboardState.SESSION_CAP) {
const firstKey = this.baselineWarmth.keys().next().value;
@@ -972,19 +973,20 @@ export class DashboardState {
// Cache-aware warmth, reconstructed from persisted ts so replay matches
// the live update() path (see baselineWarmth field + update()).
const sidR = (t as { first_user_sha8?: string }).first_user_sha8;
const prefixShaR = (t as { system_sha8?: string }).system_sha8;
const tsSec = Date.parse(t.ts) / 1000;
const warmthPrevR =
typeof sidR === 'string' && sidR.length > 0 ? this.baselineWarmth.get(sidR) : undefined;
// Same union warmth as update() (persisted ts instead of the live clock
// so replay reproduces live numbers exactly): fresh wall-clock prior OR
// cr>0. A cache-busted re-render within the TTL stays warm via the prior;
// a post-restart turn stays warm via cr. See deriveBaselineWarmth.
// so replay reproduces live numbers exactly): matching-hash fresh prior
// OR cr>0. See deriveBaselineWarmth.
const { warm: warmR, prevCacheable: prevCacheableR } = deriveBaselineWarmth(
warmthPrevR,
tsSec,
cacheable,
cr,
DashboardState.CACHE_TTL_SEC,
prefixShaR,
);
baselineInputEff = creditSaving
? computeBaselineInputEff(
@@ -1001,6 +1003,7 @@ export class DashboardState {
this.baselineWarmth.set(sidR, {
ts: tsSec,
cacheable: cacheable > 0 ? cacheable : (warmthPrevR?.cacheable ?? 0),
prefixSha: prefixShaR ?? warmthPrevR?.prefixSha,
});
if (this.baselineWarmth.size > DashboardState.SESSION_CAP) {
const firstKey = this.baselineWarmth.keys().next().value;
src/dashboard/fragments.ts
+20 -10
View File
@@ -386,8 +386,10 @@ export function renderContextMapFragment(
if (!c || (c.baselineTokens <= 0 && c.imageCount <= 0)) {
return `<div class="ctxmap"><div class="empty-note">Pick <strong>Details</strong> on a request to see exactly which parts became images and which stayed as text.</div></div>`;
}
// Cache-aware basis — identical to the recent row's As-text / Sent / Saved
// columns, so the two panels can never contradict each other. The raw
// Cache-aware billing-equivalent basis — identical to the recent row's
// As-text / Sent / Saved/lost columns. These are not raw token counts; they apply
// Anthropic's cache rates so create/read misses are visible in the comparison.
// The two panels can never contradict each other. The raw
// count_tokens ratio is cache-blind: it over-states savings whenever the
// prefix would have been a cheap cache-read, so it must NOT drive the
// headline. It survives only as a clarifying sub-line below.
@@ -439,10 +441,10 @@ export function renderContextMapFragment(
const rawPhrase =
rawShrink >= 0 ? `Raw content shrank ${rawShrink}%.` : `Raw content grew ${-rawShrink}%.`;
const headline = !showCompare
? `<strong>${kFmt(c.actualInputEff || c.realInput)}</strong> billed tokens sent`
? `<strong>${kFmt(c.actualInputEff || c.realInput)}</strong> billing-equivalent input tokens sent`
: pct >= 0
? `<span class="ctx-big">${pct}%</span> smaller — <strong>${kFmt(base)}</strong> billed tokens as ${textNoun} became <strong>${kFmt(real)}</strong> billed tokens as images`
: `<span class="ctx-big">${-pct}%</span> bigger — imaging cost <strong>${kFmt(real)}</strong> billed tokens vs <strong>${kFmt(base)}</strong> if kept as ${textNoun}`;
? `<span class="ctx-big">${pct}%</span> smaller — ${textNoun} would bill as <strong>${kFmt(base)}</strong> input tokens; images billed as <strong>${kFmt(real)}</strong>`
: `<span class="ctx-big">${-pct}%</span> bigger — images billed as <strong>${kFmt(real)}</strong> input tokens vs <strong>${kFmt(base)}</strong> for ${textNoun}`;
// Clarifying sub-line. It must match the turn's real cache state: claiming a
// 0.1× read discount on a cold turn (where the prefix actually paid the 1.25×
// create rate) — or, conversely, denying the text its warm read on a turn
@@ -502,7 +504,14 @@ export function renderRecentFragment(p: RecentPayload): string {
viewId != null
? `<a class="row-view" href="#" hx-get="/fragments/context-map?req=${viewId}" hx-target="#frag-context-map" hx-swap="innerHTML">Details →</a>`
: `<span class="muted">—</span>`;
const saved = e.session_saved_so_far_delta ?? 0;
const saved = e.session_saved_so_far_delta;
const savedCell = saved == null
? `<td class="num muted">—</td>`
: saved > 0
? `<td class="num pos">${numFmt(saved)}</td>`
: saved < 0
? `<td class="num neg">${numFmt(saved)}</td>`
: `<td class="num">0</td>`;
const imaged = e.cc_added
? `<span class="badge badge-img">image</span>`
: `<span class="badge badge-txt">text</span>`;
@@ -516,7 +525,7 @@ export function renderRecentFragment(p: RecentPayload): string {
`<td class="num">${e.cache_read != null ? numFmt(e.cache_read) : '—'}</td>` +
`<td class="num">${e.baseline_input != null ? numFmt(e.baseline_input) : '—'}</td>` +
`<td class="num">${e.actual_input != null ? numFmt(e.actual_input) : '—'}</td>` +
`<td class="num pos">${saved > 0 ? numFmt(saved) : '—'}</td>` +
savedCell +
`<td class="num">${viewLink}</td>` +
`</tr>`
);
@@ -530,9 +539,9 @@ export function renderRecentFragment(p: RecentPayload): string {
`<th>Model</th>` +
`<th title="Was this request's context compressed into an image?">Sent as</th>` +
`<th class="num" title="Tokens served from Claude's cache (cheap)">Cache hits</th>` +
`<th class="num" title="What this context would cost as plain text">As text</th>` +
`<th class="num" title="What we actually sent after imaging">Sent</th>` +
`<th class="num" title="Tokens saved on this request">Saved</th>` +
`<th class="num" title="Billing-equivalent input if kept as plain text, after cache create/read rates">As text</th>` +
`<th class="num" title="Actual billing-equivalent input after imaging, after cache create/read rates">Sent</th>` +
`<th class="num" title="As-text minus Sent; negative means imaging cost more">Saved/lost</th>` +
`<th></th>` +
`</tr></thead><tbody>${body}</tbody></table>`
);
@@ -907,6 +916,7 @@ const CSS = `
#frag-latest { overflow: auto; scrollbar-width: thin; }
th.num, td.num { text-align: right; }
td.pos { color: var(--good); font-weight: 600; }
td.neg { color: var(--bad); font-weight: 600; }
.endp { color: var(--ink); font-family: var(--mono); font-size: 11px; }
.empty-cell { color: var(--muted); text-align: center; padding: 18px; }
.pill { display: inline-block; min-width: 38px; text-align: center; font-size: 11px; font-weight: 700;
src/sessions.ts
+15 -5
View File
@@ -139,7 +139,7 @@ export async function aggregateSessions(
// Per-session warmth for the cache-aware text counterfactual, reconstructed
// from event ts as we scan in time order (mirrors DashboardState). Kept out
// of SessionSummary so it never leaks into the /api/sessions.json shape.
const warmth = new Map<string, { ts: number; cacheable: number }>();
const warmth = new Map<string, { ts: number; cacheable: number; prefixSha?: string }>();
const CACHE_TTL_SEC = 300;
// Stat sidecar sizes once up front. Looking up size per event would be
@@ -196,14 +196,22 @@ export async function aggregateSessions(
haveUsage
) {
const cacheable = ev.baseline_cacheable_tokens ?? 0;
const prefixSha = ev.system_sha8;
const tsSec = Date.parse(ev.ts) / 1000;
const prev = warmth.get(id);
// Warmth = honest union (mirrors dashboard.ts, centralised in
// deriveBaselineWarmth): warm iff a fresh same-session prior is within the
// TTL OR cr>0 witnesses a read. cr=0 no longer forces cold (the wall-clock
// leg keeps a cache-busted re-render warm), and cr>0 rescues the first
// post-restart turn that has no in-memory prior yet.
const { warm, prevCacheable } = deriveBaselineWarmth(prev, tsSec, cacheable, cr, CACHE_TTL_SEC);
// TTL AND has the same static-prefix hash, OR cr>0 witnesses a read. cr=0
// no longer forces cold when the hash matches (cache-busted image re-render),
// and cr>0 rescues the first post-restart turn with no in-memory prior.
const { warm, prevCacheable } = deriveBaselineWarmth(
prev,
tsSec,
cacheable,
cr,
CACHE_TTL_SEC,
prefixSha,
);
const baselineEff = computeBaselineInputEff(
baseline,
cacheable,
@@ -224,10 +232,12 @@ export async function aggregateSessions(
if (haveUsage) {
const tsSec = Date.parse(ev.ts) / 1000;
const cacheable = ev.baseline_cacheable_tokens ?? 0;
const prefixSha = ev.system_sha8;
const prev = warmth.get(id);
warmth.set(id, {
ts: tsSec,
cacheable: cacheable > 0 ? cacheable : (prev?.cacheable ?? 0),
prefixSha: prefixSha ?? prev?.prefixSha,
});
}
if (typeof ev.cache_read_tokens === 'number') {
tests/baseline.test.ts
+21 -1
View File
@@ -79,7 +79,11 @@ describe('computeBaselineInputEff (warmth-aware)', () => {
* prior yet. The old rule used cr ALONE (cr necessary), which mispriced both.
*/
describe('deriveBaselineWarmth (honest union: fresh prior in TTL OR cr>0)', () => {
const prev = (ts: number, cacheable: number) => ({ ts, cacheable });
const prev = (ts: number, cacheable: number, prefixSha?: string) => ({
ts,
cacheable,
...(prefixSha !== undefined ? { prefixSha } : {}),
});
it('cold when there is no prior and no observed read', () => {
expect(deriveBaselineWarmth(undefined, 1000, 5000, 0)).toEqual({ warm: false, prevCacheable: 0 });
@@ -101,6 +105,22 @@ describe('deriveBaselineWarmth (honest union: fresh prior in TTL OR cr>0)', () =
});
});
it('cold when the fresh prior has a different static-prefix hash', () => {
// Same session + inside TTL is not enough: if the system/tool prefix changed,
// the text-only request would use a different prompt-cache key too.
expect(deriveBaselineWarmth(prev(1000, 8000, 'old'), 1060, 5000, 0, CACHE_TTL_SEC, 'new')).toEqual({
warm: false,
prevCacheable: 0,
});
});
it('keeps the fresh-prior warmth when the static-prefix hash matches', () => {
expect(deriveBaselineWarmth(prev(1000, 8000, 'same'), 1060, 5000, 0, CACHE_TTL_SEC, 'same')).toEqual({
warm: true,
prevCacheable: 8000,
});
});
it('cold once the prior is older than the TTL (genuine expiry) and no observed read', () => {
expect(deriveBaselineWarmth(prev(1000, 8000), 1000 + CACHE_TTL_SEC + 1, 5000, 0)).toEqual({
warm: false,
tests/context-map.test.ts
+38 -6
View File
@@ -14,7 +14,12 @@
* 0.1× while the image was re-created cold).
*/
import { describe, it, expect } from 'vitest';
import { renderContextMapFragment, type ContextMapData } from '../src/dashboard/fragments.js';
import {
renderContextMapFragment,
renderRecentFragment,
type ContextMapData,
} from '../src/dashboard/fragments.js';
import type { RecentPayload } from '../src/dashboard/types.js';
function ctx(p: Partial<ContextMapData> = {}): ContextMapData {
return {
@@ -87,7 +92,7 @@ describe('renderContextMapFragment — cache-aware headline', () => {
ctx({ haveBaseline: false, baselineInputEff: 0, actualInputEff: 1800, baselineTokens: 7500, realInput: 1800 }),
[],
);
expect(html).toContain('billed tokens sent');
expect(html).toContain('billing-equivalent input tokens sent');
expect(html).not.toContain('% smaller');
expect(html).not.toContain('% bigger');
expect(html).toContain('no trustworthy text baseline');
@@ -115,7 +120,7 @@ describe('renderContextMapFragment — cold vs warm honesty', () => {
// headline: a real saving is still shown…
expect(html).toContain('smaller');
// …but the text side is plain "text", never "cached text".
expect(html).toContain('billed tokens as text became');
expect(html).toContain('text would bill as');
expect(html).not.toContain('as cached text');
// sub-line tells the truth about the cold turn instead of inventing 0.1×.
expect(html).toContain('No warm text cache this turn');
@@ -135,7 +140,7 @@ describe('renderContextMapFragment — cold vs warm honesty', () => {
[],
);
expect(html).toContain('smaller');
expect(html).toContain('as cached text became');
expect(html).toContain('cached text would bill as');
expect(html).toContain('after cache discounts (reads at 0.1×), same basis as the Saved column');
expect(html).not.toContain('No warm text cache this turn');
});
@@ -155,7 +160,7 @@ describe('renderContextMapFragment — cold vs warm honesty', () => {
[],
);
expect(html).toContain('bigger');
expect(html).toContain('if kept as text');
expect(html).toContain('for text');
expect(html).not.toContain('as cached text');
expect(html).toContain('No warm text cache this turn');
expect(html).not.toContain('cheap cache-read');
@@ -182,7 +187,7 @@ describe('renderContextMapFragment — cold vs warm honesty', () => {
);
// Honest loss in the headline, against the WARM ("cached text") basis.
expect(html).toContain('bigger');
expect(html).toContain('if kept as cached text');
expect(html).toContain('for cached text');
// The image-busted explanation — text warm, image cold, loss surfaced.
expect(html).toContain('re-imaged the prefix and missed the image cache');
expect(html).toContain('the text would have read warm');
@@ -190,3 +195,30 @@ describe('renderContextMapFragment — cold vs warm honesty', () => {
expect(html).not.toContain('No warm text cache this turn');
});
});
describe('renderRecentFragment — billed delta presentation', () => {
it('shows negative saved deltas instead of hiding imaging losses as missing data', () => {
const html = renderRecentFragment({
recent: [
{
ts: 0,
method: 'POST',
path: '/v1/messages',
status: 200,
compressed: true,
cc_added: 1,
cache_read: 0,
baseline_input: 7618,
actual_input: 69526,
session_saved_so_far_delta: -61908,
},
],
has_preview: false,
preview_meta: '',
} satisfies RecentPayload);
expect(html).toContain('Saved/lost');
expect(html).toContain('class="num neg">-61,908</td>');
expect(html).not.toContain('class="num pos">—</td>');
});
});
tests/dashboard-api.test.ts
+39
View File
@@ -376,6 +376,7 @@ describe('union warmth: fresh wall-clock prior OR cr>0 (no phantom savings, no h
},
cacheable: number,
sid = 'warmsess',
systemSha8 = 'stable-system',
): unknown {
return {
ts: '2026-05-19T00:00:00Z',
@@ -388,6 +389,7 @@ describe('union warmth: fresh wall-clock prior OR cr>0 (no phantom savings, no h
info: {
compressed: true,
firstUserSha8: sid,
systemSha8,
baselineProbeStatus: 'ok',
baselineTokens: 30000, // text counterfactual: full prefix + tail
baselineCacheableTokens: cacheable, // prefix up to the cache_control marker
@@ -446,6 +448,43 @@ describe('union warmth: fresh wall-clock prior OR cr>0 (no phantom savings, no h
expect(miss.session_saved_so_far_delta!).toBeLessThan(0);
});
it('prices text cold when the static prefix hash changed inside the TTL', async () => {
dash.update(
antEvt(
{
input_tokens: 100,
output_tokens: 50,
cache_creation_input_tokens: 0,
cache_read_input_tokens: 20000,
},
20000,
'hashsess',
'old-system',
) as never,
);
dash.update(
antEvt(
{
input_tokens: 100,
output_tokens: 50,
cache_creation_input_tokens: 20000,
cache_read_input_tokens: 0,
},
20000,
'hashsess',
'new-system',
) as never,
);
const recent = (await dash.serveRecent().json()) as RecentPayload;
const changed = recent.recent.at(-1)!;
// Static prefix changed, so the text-only path would create too:
// baseline = 20000*1.25 + 10000 tail = 35000, not warm 12000.
expect(changed.baseline_input).toBe(35000);
expect(changed.session_saved_so_far_delta).toBe(9900);
});
it('still prices a genuine warm turn warm (cr>0 reads the prefix cheaply)', async () => {
// Prime, then a real warm turn: cache_read > 0, small growth.
dash.update(
tests/sessions.test.ts
+36
View File
@@ -261,6 +261,42 @@ describe('aggregateSessions', () => {
// The old cr-alone code booked the second turn as +31250 → a 62500 overclaim.
expect(s.tokensSavedEst).toBe(30_300);
});
it('does not treat a fresh prior as warm when the static prefix hash changed', async () => {
writeEvents(tmp, [
ev({
first_user_sha8: 'dddddddd',
ts: '2026-05-19T00:00:00.000Z',
compressed: true,
baseline_tokens: 30_000,
baseline_cacheable_tokens: 20_000,
system_sha8: 'old-system',
input_tokens: 100,
output_tokens: 50,
cache_create_tokens: 0,
cache_read_tokens: 20_000,
}),
ev({
first_user_sha8: 'dddddddd',
ts: '2026-05-19T00:01:00.000Z',
compressed: true,
baseline_tokens: 30_000,
baseline_cacheable_tokens: 20_000,
system_sha8: 'new-system',
input_tokens: 100,
output_tokens: 50,
cache_create_tokens: 20_000,
cache_read_tokens: 0,
}),
]);
const { sessions } = await aggregateSessions(tmp);
const s = sessions.get('dddddddd')!;
// Turn 1: baseline full-reuse via cr = 20000*0.1 + 10000 = 12000;
// actual = 100 + 20000*0.1 = 2100; saved = 9900.
// Turn 2: static hash changed, so text is cold too: baseline = 35000;
// actual = 25100; saved = 9900. Old wall-clock-only warmth booked -13100.
expect(s.tokensSavedEst).toBe(19_800);
});
});
// ---- filter + list ---------------------------------------------------------