From 9ecd79909256d18732d8f50ea1c08494bb900bcc Mon Sep 17 00:00:00 2001 From: local Date: Thu, 21 May 2026 14:34:06 -0400 Subject: [PATCH] =?UTF-8?q?feat(transform):=20lower=20SLAB/HISTORY=5FCHARS?= =?UTF-8?q?=5FPER=5FTOKEN=202.5=20=E2=86=92=202.0=20for=20Opus=204.7=20tok?= =?UTF-8?q?enizer?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit N=391 Opus 4.7 measured rows (last 7d) show real chars-per-token = 1.91 (avg_outgoing_text_chars=231,925 / avg_real_input_tokens=115,893). The older 2.5 constant was calibrated on Opus 4.5/4.6 and over-estimated cpt by ~30%, under-counting text-token cost and causing the break-even gate to reject most history-collapse opportunities (283/391 = 72% of measured ok rows came back history_reason='not_profitable'). Resize one render.test.ts fixture (6060 → 4848 chars) so the 'low cpt unlocks a previously-rejected slab' assertion still exercises the intended before/after edge under the new default. --- src/core/transform.ts | 78 ++++++++++++++++++++----------------------- tests/render.test.ts | 14 +++++--- 2 files changed, 46 insertions(+), 46 deletions(-) diff --git a/src/core/transform.ts b/src/core/transform.ts index 893f8bb..8b68f23 100644 --- a/src/core/transform.ts +++ b/src/core/transform.ts @@ -178,60 +178,56 @@ const DEFAULTS: Required = { * mix; tool_result content is typically code-shaped. */ const CHARS_PER_TOKEN = 4; -/** Empirical chars-per-token for the *system slab + tool docs* path. +/** Empirical chars-per-token for the *image-slab* path. * - * Source: N=354 production cold-miss `count_tokens` baselines on - * Claude Code (pixelpipe events.jsonl, 2026-05-18 → 2026-05-20): - * body cpt distribution — median 1.17, p95 2.5, MAX 2.62. + * Updated 2026-05-21 from 2.5 → 2.0 on the basis of N=391 production + * rows on Opus 4.7 (last 7 days, baseline_probe_status='ok'): * - * System slabs are JSON-dense (tool definitions, schemas, structured - * prompts) so they sit at cpt ~1.2, NOT the English-prose 4. Using cpt=4 - * for this call site told the gate text was 3.4× cheaper than reality, - * silently rejecting every profitable slab compression we've seen since - * the row-aware gate landed. + * avg_outgoing_text_chars = 231,925 + * avg_real_input_tokens = 115,893 (input + cache_created + cache_read) + * => observed cpt = 1.91 * - * Safety: 2.5 is the **upper bound** of observed real cpts (max=2.62 - * with one sample, p95=2.5). Picking the upper bound keeps the prime- - * directive guarantee intact — the text-token estimate is a LOWER bound - * on real text cost, so any `imageCost < textTokens` decision is also - * `imageCost < realTextCost`. If a slab in the wild ever lands above - * cpt=2.5, the gate would still under-bill text by < 5% and the worst - * case is a marginal-loss compression, not a runaway. + * Opus 4.7 ships a new tokenizer; the older 2.5 value was calibrated on + * Opus 4.5/4.6 text and now systematically OVER-estimates chars-per-token + * by ~30%, which UNDER-estimates the real text-token cost the gate is + * comparing image cost against. The result: the gate rejects compressions + * that would actually be net wins. 2.0 brings the gate's text-token + * estimate close to reality while still rounding slightly conservative + * versus the observed mean (1.91), so a marginal-shape outlier slab can + * still fail the gate without becoming a runaway loss. * * Why this is slab-specific and NOT a global default: reminders and * tool_result content have unknown shape (could be raw English prose * with cpt~4). Leaving those at CHARS_PER_TOKEN=4 preserves the * conservative bias where shape isn't known a priori. */ -export const SLAB_CHARS_PER_TOKEN = 2.5; +export const SLAB_CHARS_PER_TOKEN = 2.0; /** Empirical chars-per-token for the *history compression* path. * - * Source: same N=354 production telemetry as SLAB_CHARS_PER_TOKEN, plus - * a tighter sub-sample of N=10 events where the history collapse was - * rejected as `not_profitable` (history_reason="not_profitable" in - * events.jsonl, 2026-05-18 → 2026-05-20): - * body cpt on rejected history events — min 1.08, median 1.09, MAX 1.10. + * Updated 2026-05-21 from 2.5 → 2.0 on the same Opus 4.7 telemetry as + * SLAB_CHARS_PER_TOKEN (N=391, observed cpt=1.91). History content is + * even denser than the slab path on average (tool_use JSON dominates + * Claude Code sessions, body cpt observed ~1.09 on rejected events), + * so the 2.5 → 2.0 move here is doubly conservative. * - * History content shape: collapsed-prefix turns from the conversation = - * past tool_use args (JSON, very dense), tool_result blocks (variable), - * assistant/user text (prose, less dense). In tool-heavy Claude Code - * sessions the JSON dominates and the observed body-cpt sits at ~1.09 — - * even denser than the slab path's 1.17 median. + * Diagnostic that drove this change: 283/391 = 72% of measured Opus 4.7 + * pixelpipe rows in the last 7 days carried + * pixelpipe_history_reason='not_profitable'. The break-even gate was + * rejecting most history-collapse opportunities because it was + * comparing real image costs against a 30%-under-counted text cost. * - * Why 2.5 (not lower despite tighter empirical data): a chat-heavy - * session with predominantly user/assistant prose could push history - * content closer to the English-prose cpt~4 range. We have no telemetry - * for those sessions yet (chat-only Claude Code is rare). 2.5 = slab's - * empirically-justified upper bound, applied here to keep both call - * sites symmetric and to absorb chat-shape uncertainty. If a real - * chat-only session ever drives observed cpt above 2.5 we'll see it - * in the dashboard before any net-loss compression lands. + * Prediction grading (one session out): 'not_profitable' rows should + * drop below 40% of ok-status measured Opus 4.7 rows; the 'collapsed' + * count should rise. If 'not_profitable' stays >55%, the gate is being + * rejected for a non-cpt reason (horizon, prefix shape) and constants + * alone won't fix it. * - * Safety: at cpt=2.5 the gate's text-token estimate (textLen / 2.5) is - * a LOWER bound on real text cost whenever real cpt ≤ 2.5 (= every - * history sample we've ever observed). Image-cost-vs-estimated-text - * passing therefore implies image-cost-vs-real-text passing. */ -export const HISTORY_CHARS_PER_TOKEN = 2.5; + * Safety: at cpt=2.0 the gate's text-token estimate (textLen / 2.0) is + * a LOWER bound on real text cost whenever real cpt ≤ 2.0. The current + * observed cpt is 1.91, comfortably under. Chat-only sessions could in + * theory drive cpt above 2.0 — if that ever lands in production, we'll + * see it in the dashboard before any net-loss compression. */ +export const HISTORY_CHARS_PER_TOKEN = 2.0; /** Empirical per-image cost at numCols=1. Source: dashboard.ts measurement * trace. Kept here as a constant rather than imported from dashboard.ts @@ -1975,7 +1971,7 @@ export async function transformRequest( // count after wrapping. // History cpt is empirically ~1.09 (N=10 rejected-events sample) — JSON- // dense like the slab, so use the same conservative upper-bound cpt - // baked into HISTORY_CHARS_PER_TOKEN=2.5. Host override (opts.charsPerToken) + // baked into HISTORY_CHARS_PER_TOKEN=2.0. Host override (opts.charsPerToken) // wins if the dashboard ever feeds back a live empirical fit. // Same discriminator as the slab path: check the *raw* `opts` so a host // that genuinely wants `4` can pin to it without colliding with the merged diff --git a/tests/render.test.ts b/tests/render.test.ts index 938046e..ad509b4 100644 --- a/tests/render.test.ts +++ b/tests/render.test.ts @@ -1831,12 +1831,16 @@ describe('transform', () => { }); it('TransformOptions.charsPerToken: low value unlocks a previously-rejected slab end-to-end', async () => { - // Dense 6060-char slab — rejected at default cpt=4, accepted at cpt=1.14. - // Forced to numCols=1 (single-col) so the cpt override does the flipping - // on its own; multi-col adds a separate 10% safety margin (see the - // numCols=2 regression test above). + // Dense 4848-char slab — rejected at default SLAB_CHARS_PER_TOKEN=2.0 + // (text=2424 tok < single-col image cost 2500), accepted at cpt=1.14 + // (text=4252 tok > 2500). Forced to numCols=1 so the cpt override does + // the flipping on its own; multi-col adds a separate 10% safety margin + // (see the numCols=2 regression test above). Resized 2026-05-21 when + // SLAB_CHARS_PER_TOKEN moved 2.5 → 2.0; the 6060-char fixture now + // compresses under the new default and stopped exercising the intended + // before/after edge. const line = 'A'.repeat(100) + '\n'; - const slab = line.repeat(60); // 6060 chars + const slab = line.repeat(48); // 4848 chars const req = JSON.stringify({ model: 'claude-3-5-sonnet', messages: [{ role: 'user', content: 'hi' }],