# Prompt-Caching Alignment And Honest Savings Math

This doc answers two questions:

1. How pxpipe stays aligned with Anthropic prompt caching when it rewrites bulky context into images.
2. How pxpipe reports savings without counting the provider cache discount as a pxpipe win.

Source of truth in code: `src/core/transform.ts` for the cache-aligned rewrite and `src/core/baseline.ts` for the accounting.

---

## Anthropic Prompt Cache Basics

Anthropic prompt caching is prefix-based:

- The cache key is derived from the exact rendered prompt bytes up to each `cache_control` breakpoint.
- Render order is `tools` -> `system` -> `messages`.
- A breakpoint is a `"cache_control": {"type": "ephemeral"}` marker on a content block.
- The response usage block reports `input_tokens`, `cache_creation_input_tokens`, and `cache_read_input_tokens`.
- Total prompt tokens for the actual request are `input_tokens + cache_creation_input_tokens + cache_read_input_tokens`.

Pricing relative to base input rate:

| bucket | meaning | rate |
|---|---|---:|
| `input_tokens` | uncached input | `1.0x` |
| `cache_creation_input_tokens` (`cc`) | cache write | `1.25x` |
| `cache_read_input_tokens` (`cr`) | cache read | `0.1x` |
| output | model reply | `5x` input on Fable 5 |

A stable prefix is cheap after it is cached, but a prefix that changes every turn repeatedly pays the write premium. pxpipe's cache-alignment work exists to keep the image prefix stable.

---

## Cache-Aligned Rewrite

Claude Code sends a large stable prefix: system prompt, tool docs, reminders, and older history. It also sends a small per-turn tail: the current user message and dynamic runtime context.

pxpipe rewrites only the bulky, cacheable parts into images. The dynamic parts stay as text so they do not pollute the image cache key.

The key invariant:

> pxpipe does not add new cache-control markers. It relocates the caller's existing marker onto the last image block produced from the same logical content.

That keeps the breakpoint at the end of the rewritten stable content. The provider then caches the image prefix exactly as it would have cached the text prefix, just with fewer input tokens under that cache entry.

The transformed request shape is:

```text
system:
  billing line / dynamic context / other text-only system content

messages[0] user:
  image block
  image block
  image block + cache_control
  [End of rendered context.]
  original user content / live tail
```

Images must be placed in a user message because Anthropic does not accept images in the `system` field.

---

## Gate And One-Time Burn

The raw compression gate compares image token cost with text token cost. For Anthropic, image cost is estimated from pixel area and text cost comes from the configured chars/token estimate for the relevant bucket.

Switching modes can also burn a warm cache. If a text prefix is warm and pxpipe flips to images, the first image turn may pay a cache write. If an image prefix is warm and pxpipe flips back to text, the text path may pay the write. The symmetric burn terms model that cost:

```text
burnImageSide = priorWarmTokens      * (1.25 - 0.10)
burnTextSide  = priorWarmImageTokens * (1.25 - 0.10)

compress iff imageTokens + burnImageSide < textTokens + burnTextSide
```

This is separate from dashboard savings. The gate decides whether to transform. The dashboard reports what the transformed request actually cost against the measured text counterfactual.

---

## Savings Accounting

The cache discount is provider behavior, not pxpipe savings. To avoid counting cache as savings, both sides use the same observed cache state:

- If the actual request has `cr > 0`, the imagined text baseline is priced warm too.
- If the actual request has `cr === 0`, the imagined text baseline is priced cold too.

pxpipe does not infer a warm text baseline from wall-clock TTL alone. The text request does not actually exist, so cache warmth for that counterfactual is based only on the real request's server-reported cache read.

For each `/v1/messages` request, pxpipe records three measurements:

1. Actual upstream usage from the transformed request: `input_tokens`, `cc`, `cr`, and output.
2. `baseline_tokens`: `/count_tokens` on the original body before compression.
3. `baseline_cacheable_tokens`: `/count_tokens` on the original body truncated at the last `cache_control` marker.

The actual input cost is:

```text
actual_eff = input_tokens + cc * 1.25 + cr * 0.10
```

The text baseline first splits the measured text tokens:

```text
cacheable = min(baseline_cacheable_tokens, baseline_tokens)
coldTail  = baseline_tokens - cacheable
```

If the actual request is cold (`cr === 0`):

```text
baseline_eff = cacheable * 1.25 + coldTail
```

If the actual request is warm (`cr > 0`):

```text
reused = min(prevCacheable, cacheable)
grown  = cacheable - reused

baseline_eff = reused * 0.10 + grown * 1.25 + coldTail
```

`prevCacheable` is used only after `cr > 0` proves a warm read. It refines how much of the text baseline was reused vs newly grown. If there is no completed same-session prior with the same static-prefix hash, pxpipe assumes full reuse for the text baseline: `prevCacheable = cacheable`. That is conservative for savings because it makes the text baseline cheaper.

Replay uses request start time (`ts - duration_ms`) to avoid overlapping requests refining each other's `prevCacheable` split before the earlier request had completed.

The savings number is then:

```text
savings_eff = baseline_eff - actual_eff
```

This can be negative when imaging actually costs more under the same cache state. Negative rows are not hidden or floored.

Rows without a trustworthy cacheable-prefix probe contribute zero savings rather than guessing. Uncompressed rows also contribute zero savings.

---

## Worked Examples

Assume a request whose original text baseline is `30,000` input tokens. Of those, `28,000` are before the cache-control marker, so `coldTail = 2,000`. pxpipe renders that prefix to `3,000` image tokens.

### Warm Request

The actual request reports `input_tokens = 2,000`, `cc = 1,000`, `cr = 3,000`. A prior completed row shows `prevCacheable = 27,000`.

```text
Text baseline:
  reused = min(27000, 28000) = 27000
  grown  = 28000 - 27000     = 1000
  baseline_eff = 27000*0.10 + 1000*1.25 + 2000
               = 2700 + 1250 + 2000 = 5950

Actual image request:
  actual_eff = 2000 + 1000*1.25 + 3000*0.10
             = 2000 + 1250 + 300 = 3550

Savings = 5950 - 3550 = 2400
```

The cache read discount applies to both sides. The win is that the warm image prefix is `3,000` tokens while the warm text prefix would have been `27,000` reused text tokens plus `1,000` grown text tokens.

### Cold Request

The actual request reports `input_tokens = 2,000`, `cc = 3,000`, `cr = 0`.

```text
Text baseline:
  baseline_eff = 28000*1.25 + 2000
               = 35000 + 2000 = 37000

Actual image request:
  actual_eff = 2000 + 3000*1.25
             = 2000 + 3750 = 5750

Savings = 37000 - 5750 = 31250
```

Both sides are cold. pxpipe is not credited for cache; it is credited because the cold image write is much smaller than the cold text write.

---

## Reproducing The Dashboard Math

Every row in `~/.pxpipe/events.jsonl` carries the fields needed to reproduce the input-side savings:

- `baseline_tokens`
- `baseline_cacheable_tokens`
- `input_tokens`
- `cache_create_tokens`
- `cache_read_tokens`
- `first_user_sha8`
- `system_sha8`
- `ts`
- `duration_ms`

Walk rows in completion order. For each session (`first_user_sha8`), keep the latest completed row's `baseline_cacheable_tokens`, `system_sha8`, and completion timestamp. For the current row:

1. Set `warm = cache_read_tokens > 0`.
2. If `warm` and the previous row completed before this request started and has the same `system_sha8`, use its `baseline_cacheable_tokens` as `prevCacheable`.
3. If `warm` but no usable prior exists, use this row's `cacheable` as `prevCacheable`.
4. If not `warm`, use `prevCacheable = 0`.
5. Compute `baseline_eff`, `actual_eff`, and `baseline_eff - actual_eff` with the formulas above.

The live dashboard and replay path both use `deriveBaselineWarmth`, `computeBaselineInputEff`, and `computeActualInputEff` from `src/core/baseline.ts`, so the UI and session summaries use the same math.

---

## Summary

pxpipe stays cache-aligned by replacing stable text context with stable image context and relocating the caller's existing cache marker to the end of the rewritten content. Savings are measured by comparing the real transformed request with a `/count_tokens` text counterfactual under the same observed cache state. If the actual request read cache, both sides are warm. If it did not, both sides are cold. Therefore the provider cache discount is not counted as pxpipe savings; the reported savings are only the token reduction from text to images.
