BFlat 5ea84a8f4c Cached prompt tokens from OpenAI-style APIs are counted once (#337) ...

OpenAI-style APIs report `prompt_tokens` including the cached part (`prompt_tokens_details.cached_tokens`). We stored the whole `prompt_tokens` as `input_tokens` and the cached part again as `cache_read_tokens`. The context budget and the cost meter add the two, so cached tokens counted twice: a 100,000-token prompt with 65,536 cached read as 165,536, crossing the 150k budget and summarising early, and it was charged both rates.

`input_tokens` is now `prompt_tokens` minus the cached tokens, as Anthropic reports it, so every consumer counts each token once.

**Verified:** a new stream test with the reporter's numbers (100,000 prompt, 65,536 cached, 10 output): input 34,464, cache reads 65,536. `cargo test -p craft-anything-ai` (186), clippy and fmt pass.

Fixes #252

2026-11-02 15:19:45 +07:00

..

src

Cached prompt tokens from OpenAI-style APIs are counted once (#337)

2026-11-02 15:19:45 +07:00

Cargo.toml

Craft Anything

2026-10-31 10:52:14 -04:00