mirror of
https://github.com/teamchong/pxpipe.git
synced 2026-07-22 02:02:51 +02:00
teamchong
dfd3bf514f
The baseline_tokens probe alone reports the COLD cost of the unproxied
body — but the user's request actually carries `cache_control` markers
that would have cached on the unproxied path too. Comparing cold-
baseline against warm-actual was an apples-to-oranges inflation of
saved%.
Fix: fire a SECOND count_tokens probe on the body TRUNCATED at the last
`cache_control` marker. Difference between the two is (cacheable_prefix,
cold_tail). Dashboard weights the prefix by the SAME cache class the
actual response landed in:
cacheable = baseline_cacheable_tokens or 0
cold_tail = baseline_tokens − cacheable
weight = cr>0 ? 0.10 (warm hit — both paths cache)
: cc>0 ? 1.25 (cold create — both paths build cache)
: 1.0 (no caching — marker rejected or absent)
baseline_input_eff = cacheable·weight + cold_tail·1.0
actual_input_eff = input + cache_create·1.25 + cache_read·0.10
saved = baseline_input_eff − actual_input_eff
Now the headline number is the exact unproxied counterfactual, not a
cold-every-time approximation. No estimation, no α, no assumptions —
every input is a measurement.
Wiring:
- src/core/proxy.ts: buildCountTokensCacheablePrefix() walks messages
backward → system backward → tools backward (reverse cache order) for
the last cache_control marker, truncates, attaches a sentinel user
message when needed. Second probe fires in parallel; null/missing
leaves the field absent.
- src/core/transform.ts + tracker.ts: new baselineCacheableTokens /
baseline_cacheable_tokens fields, emitted only when > 0.
- src/dashboard.ts: per-event cache-aware math in update() and
replay(); Totals renamed to baselineInputWeighted; /proxy-stats
payload field is `baseline_input_weighted`.
- src/sessions.ts: per-session tokensSavedEst uses the same formula so
session and global rollups agree.
Tests: 2 new in proxy-usage covering (a) body with cache_control →
two probes fire with different bodies and both numbers land on info,
(b) cacheable-prefix probe failure doesn't poison the full probe.
260→262 tests, all green; typecheck and build clean.