Reapply "fix(render): fit Anthropic page to 1568×728 (~1.15 MP) for WYSIWYG glyphs"

This reverts commit 6eaa8bfcd0.

docs/LEGIBILITY-AUDIT-2026-07-01.md

@@ -0,0 +1,110 @@
+# Imaged-text legibility audit — 2026-07-01
+
+Motivated by a demo failure: the pxpipe arm of `demo/effective-context` read all
+files but could not report the `AUDIT-ZX9` line count, and hedged that imaged text
+was uncertain. This audit measured **why**, with data, and shipped the fixes that
+follow from it.
+
+## TL;DR
+
+- The Anthropic API **downscales every image** to fit BOTH long-edge ≤1568 px AND
+  ~1.15 MP, then bills ≈ px/750 (per-image). Our old 1932×1932 page was billed at
+  the cap but resampled **0.555×** — 5×8 glyphs reached the vision encoder at
+  ~2.8×4.4 px. We were paying full price for pixels destroyed in transit.
+- **Shipped:** page geometry clamped to **1568×728 = 1.14 MP** (WYSIWYG: billed/actual
+  pixel ratio 3.25 → 1.04). No token-cost change per image; 614/614 tests green.
+- Reading exact strings off even **crisp** 5×8 tops out at **63%** (24-token blind
+  test). Every miss is one of two classes the glyph-confusability matrix predicts:
+  **case-normalization** (camelCase) and **single-glyph substitution** (hex/num).
+- **7 of 9 misses are already covered by the factsheet** (SHAs, numbers ride as
+  text). The 2 residual misses are camelCase code identifiers — not a factsheet shape.
+- **Answer to "factsheet vs. RAG-escape":** both, split on a *sparsity* axis. Sparse
+  precision (a few ids/page) → factsheet. Dense precision (every symbol in a code
+  dump, >64 distinct) → the factsheet's 64-token budget can't hold it; that needs the
+  re-fetch path. The scaffolding (`RecoverableBlock`) exists but is not yet wired to a
+  model-callable tool.
+
+## Method
+
+`count_tokens` sweep (claude-sonnet-4-5) on synthetic PNGs at controlled dims;
+14,935-row regression on real proxy traffic (`~/.pxpipe/events.jsonl`) for the
+unresized slope; pixel-level Hamming matrix over the Spleen 5×8 atlas; and a 24-token
+blind read where transcriptions were committed *before* revealing ground truth.
+
+## 1. API resize (the free-fidelity lever)
+
+| dims | px | px/750 | billed | verdict |
+|---|---|---|---|---|
+| 1072×1072 | 1.15 M | 1532 | 1525 | at cap |
+| 1092×1092 | 1.19 M | 1589 | 1525 | CLAMPED |
+| 1568×1568 | 2.46 M | 3278 | 1525 | CLAMPED |
+| 1928×1928 (old page) | 3.72 M | 4956 | 1525 | CLAMPED (resampled 0.555×) |
+| **1568×728 (new page)** | **1.14 M** | **1522** | **1460** | **linear — WYSIWYG** |
+
+Cap is **per-image** (2×1092 in one request billed 3049 ≈ 2×1525). Sub-cap images pass
+through unresized (real-traffic slope 733 px/tok, ≈ the documented 750).
+
+**Cost nuance (not free):** the encoder caps at ~1.15 MP/image regardless, so per-image
+cost is ~unchanged, but the new page holds 28,080 chars vs the old 92,160 → ~3.3× more
+images for the same corpus. The old "16× vs text" compression was **partly fictional**:
+it compressed by discarding pixels at the API boundary as silent blur. True legible rate
+is ~5× vs text. The change doesn't add cost — it makes the model pay *visibly* for
+legibility it was already being charged for but never received.
+
+## 2. Glyph confusability (Spleen 5×8, 94 printable ASCII)
+
+45 of 94 chars have a nearest neighbor at Hamming ≤ 3 px (of 40). Worst:
+`,~;` `.~:` `H~K` at d=1; `0~O` `3~8` `5~S` `6~8` `8~B` `U~u` `V~v` `W~w` `X~x` at d=2.
+`H~K` at d=1 is a font defect (glyph-surgery candidate). Downscaling 0.555× (old page)
+collapses many d=2 pairs toward d=0 — indistinguishable.
+
+## 3. Blind read accuracy (crisp 5×8, 24 tokens)
+
+**15/24 = 63% exact-match.** By class: camel 6/8, num 6/8, **hex 3/8 (38%)**.
+
+Miss taxonomy — 100% predicted by the matrix:
+- **2/2 camel misses are case-only** (`tokenLedgerShard`→`tokenledgerShard`): the S/s
+  class. This is the exact error that opened the session (`extractFactSheetTokens`
+  misread as `extractFactsheetTokens`).
+- **7/9 total misses are hex/num single- or double-glyph substitution**
+  (`6↔8`, `5↔3`, `7↔4`) — the d=2 matrix pairs.
+
+Reading harder cannot fix this: 5×8 is at the information floor for these pairs, and the
+decoder fills ambiguity with a *confident, plausible* guess, not an "unclear" flag.
+
+## 4. Where the factsheet lands
+
+Of the 9 missed tokens, **7 are tier-0 factsheet shapes** (SHA/number) already emitted
+as verbatim text next to the image — so the image misread is irrelevant in production.
+The **2 residual** misses are camelCase identifiers, which are *not* a factsheet shape.
+
+This is the empirical case for the factsheet: for sparse precision-critical tokens it
+converts a coin-flip-per-glyph read into a 100% text quote, at ~5% of source chars.
+
+## Recommendations
+
+1. **Done — clamp to 1568×728.** Free fidelity; ratio 3.25→1.04; tests green.
+2. **Done earlier — ticket-ID factsheet pattern + `×N` counts** (`AUDIT-ZX9`, `CVE-…`).
+   Directly fixes the demo failure; validated by the `tick` class here.
+3. **Recommended, not yet shipped — camelCase in the factsheet, carefully.** A dense
+   code page can contain **>64 distinct** camelCase symbols, so a blind add would flood
+   the 64-token budget and be lossy. Scope to ≥2 case-boundaries and let it sit at
+   tier-1 (below SHAs/nums) so it can never evict critical tokens. Measure budget impact
+   on a real code corpus first.
+4. **Recommended — expose the re-fetch path.** `RecoverableBlock` already records
+   `rec_<sha>` + original text + provenance for every imaged block. Wiring it to a
+   model-callable "rehydrate this region as text" tool is the real answer for
+   **dense** exact-recall (code files, large tables) that no fixed-size sidecar can
+   hold. This is the "put a glass over it" mechanism, and it's half-built.
+5. **Optional — glyph surgery** on `H~K` (d=1 defect) and case-contrast for the S/s
+   class. Zero token cost, bounded upside; do it in the same eval harness as (3).
+
+## Verdict on "is the factsheet the way, or should there be a RAG escape?"
+
+Neither alone. The data draws a clean line on token **sparsity**:
+- **Sparse** precision (few ids per page): factsheet. Proven — covers 7/9 misses.
+- **Dense** precision (every symbol in a dump): exceeds any sidecar budget → re-fetch
+  (`RecoverableBlock` → a rehydrate tool). Scaffolded, not yet exposed.
+
+The image carries gist and structure (which survive fine); exactness rides a different
+channel chosen by how much of it there is.

src/core/export.ts

@@ -33,7 +33,7 @@ import {
 export const DEFAULT_EXPORT_MODEL = 'claude-sonnet-4-5';
 // Chars-per-token for the reporting estimate now lives in transform.ts as
 // REPORT_CHARS_PER_TOKEN (single source of truth for all token-estimate constants).
-/** Default column width — dense content mode (384 cols = 1928 px). */
+/** Default column width — dense content mode (312 cols = 1568 px). */
 export const DEFAULT_EXPORT_COLS: number = DENSE_CONTENT_COLS;
 
 // ---------------------------------------------------------------------------

src/core/gpt-model-profiles.ts

@@ -16,7 +16,15 @@
  *   PXPIPE_GPT_PROFILES='{"gpt-5.6":{"stripCols":176}}'   # widen only
  */
 
-import { MAX_HEIGHT_PX } from './render.js';
+/**
+ * GPT strip height, DECOUPLED from render.ts's MAX_HEIGHT_PX (which is Anthropic's
+ * 1568-edge / ~1.15 MP clamp). OpenAI's pre-tokenize resize is different: fit within
+ * 2048×2048, then shortest side → 768. A 768-px-wide portrait strip up to 2048 px tall
+ * survives un-resampled, so GPT keeps the taller page. Every built-in cost number below
+ * (1190 / 1445 / 2372 / 1464 / 630 …) was calibrated at this height — do not re-link to
+ * the Anthropic constant.
+ */
+export const GPT_MAX_HEIGHT_PX = 1932;
 
 /** Image-token cost model (mirrors OpenAI's mandatory pre-tokenize resize). */
 export type GptVisionCost =
@@ -38,7 +46,7 @@ export interface GptModelProfile {
 export const DEFAULT_GPT_STRIP_COLS = 152;
 
 const C = DEFAULT_GPT_STRIP_COLS;
-const H = MAX_HEIGHT_PX;
+const H = GPT_MAX_HEIGHT_PX;
 
 /**
  * Conservative fallback for unrecognized models: tile 85/170 over-states cost,

src/core/library.ts

@@ -165,7 +165,7 @@ export interface RenderTextToImagesOptions {
   readonly maxCharsPerImage?: number;
   /** Render style. Default DENSE_RENDER_STYLE (bare 5×8 cell, anti-aliased). */
   readonly style?: RenderStyle;
-  /** Max page height in px. Default MAX_HEIGHT_PX (1932). */
+  /** Max page height in px. Default MAX_HEIGHT_PX (728 — Anthropic 1568-edge / ~1.15 MP safe). */
   readonly maxHeightPx?: number;
 }
 

src/core/openai-history.ts

@@ -21,7 +21,8 @@
  * HistoryTurn list and the planner/renderer are shared.
  */
 
-import { renderTextToPngs, reflow, neutralizeSentinel, MAX_HEIGHT_PX, type RenderedImage } from './render.js';
+import { renderTextToPngs, reflow, neutralizeSentinel, type RenderedImage } from './render.js';
+import { GPT_MAX_HEIGHT_PX } from './gpt-model-profiles.js';
 import { countTokens as o200kCountTokens } from 'gpt-tokenizer/encoding/o200k_base';
 
 /** Portrait-strip width for GPT history images. Mirrors GPT_STRIP_COLS in
@@ -101,7 +102,9 @@ export const GPT_HISTORY_DEFAULTS: GptHistoryOptions = {
   collapseChunk: 10,
   freezeChunk: 10,
   sectionTokens: 2000,
-  maxHeightPx: MAX_HEIGHT_PX,
+  // GPT path: OpenAI's resize bounds (2048-bbox / 768 short side) permit the tall
+  // strip — do NOT re-link to render.ts MAX_HEIGHT_PX (Anthropic's 1568/1.15 MP clamp).
+  maxHeightPx: GPT_MAX_HEIGHT_PX,
   maxImages: GPT_HISTORY_MAX_IMAGES,
   reflow: true,
 };

src/core/render.ts

@@ -23,21 +23,26 @@ import {
 } from './atlas-gray.js';
 import { encodeGrayPng, encodeRgbPng } from './png.js';
 
-/** Page-height ceiling. 1932 px → 384-col page = 69×69 = 4761 tokens, under both Fable/Opus 4.8
- *  limits (≤4784 tokens AND ≤2000 px/side for >20-image requests). Exported for transform.ts math. */
-export const MAX_HEIGHT_PX = 1932;
-/** Char budget for the static slab (system + tools + CLAUDE.md). ~50k leaves headroom for
- *  soft-wrap and dropped chars; pages stay well under the 1932 px ceiling at cols=313. */
-export const READABLE_CHARS_PER_IMAGE = 50000;
-/** Char budget for dense content (tool output, collapsed history). 384 cols × 240 rows = 92160 chars
- *  fills the full ~1932×1932 ceiling. NOTE: verbatim recall of imaged text is unreliable at any size. */
-export const DENSE_CONTENT_CHARS_PER_IMAGE = 92160;
-export const DENSE_CONTENT_COLS = 384;
+/** Page-height ceiling. Measured (2026-07-01, count_tokens sweep, claude-sonnet-4-5 — see
+ *  /tmp/pxexp/LEVER1-findings.md): the API downscales any image to fit BOTH long-edge ≤1568
+ *  AND ~1.15 MP (≈1,143,750 px), then bills ≈ px/750 (≈1525 tok cap, applied per-image).
+ *  The old 1932×1932 page was billed at cap but resampled 0.555× → 5×8 glyphs reached the
+ *  encoder at ~2.8×4.4 px. New page shape 1568×728 = 1,141,504 px fits both bounds →
+ *  WYSIWYG for the vision encoder (also satisfies ≤2000 px/side for >20-image requests). */
+export const MAX_HEIGHT_PX = 728;
+/** Char budget for the static slab (system + tools + CLAUDE.md). Matches physical page
+ *  capacity at 312 cols × 90 rows so image-count estimates track real pagination. */
+export const READABLE_CHARS_PER_IMAGE = 28080;
+/** Char budget for dense content (tool output, collapsed history). 312 cols × 90 rows = 28080
+ *  chars fills the 1568×728 page. NOTE: verbatim recall of imaged text is unreliable at any size. */
+export const DENSE_CONTENT_CHARS_PER_IMAGE = 28080;
+export const DENSE_CONTENT_COLS = 312;
 /** Bare 5×8 cell (no padding). A/B showed 5×8 beats 7×10 on dense JSON (4/5 vs 3/5 reads, 42% fewer tokens).
  *  Revert to {cellWBonus:2, cellHBonus:2} if misread rates rise. */
 export const DENSE_RENDER_STYLE: RenderStyle = { cellWBonus: 0, cellHBonus: 0, aa: true };
-/** Default columns for the static slab. 313 × 5 px + 8 px pad = 1573 px — under the 1932 px ceiling. */
-const DEFAULT_COLS = 313;
+/** Default columns for the static slab. 312 × 5 px + 8 px pad = 1568 px — exactly the API's
+ *  long-edge bound (313 cols = 1573 px would trigger a 0.997× resample, blurring every glyph). */
+const DEFAULT_COLS = 312;
 /** Horizontal padding (left + right each), px. Exported for transform.ts token-cost math. */
 export const PAD_X = 4;
 /** Vertical padding (top + bottom each), px. Exported for transform.ts token-cost math. */
@@ -732,9 +737,9 @@ export async function renderTextToPngs(
 // OCR column ordering is the risk — gated behind an opt-in flag pending empirical eval.
 
 const GUTTER_CELLS = 4;
-// Width is capped at 1932 px, not 2000 px: a 1932-tall page = 69 patches, so width must
-// also be ≤69 patches (4761 tokens ≤ 4784). A 2000-px width → 72×69 = 4968 > 4784 → REJECTED.
-const MAX_WIDTH_PX = 1932;
+// Width hard-capped at the API's long-edge bound: anything wider is resampled server-side
+// (measured 2026-07-01: fit-within 1568-edge AND ~1.15 MP, then ≈px/750 billing).
+const MAX_WIDTH_PX = 1568;
 
 const GUTTER_DIVIDER_INK = 64; // pre-invert → 191 post-invert: light gray column separator
 const GUTTER_DIVIDER_INSET_PX = 2; // keep divider clear of padding rows

tests/paging.test.ts

@@ -28,14 +28,15 @@ import {
   DENSE_CONTENT_CHARS_PER_IMAGE,
   DENSE_CONTENT_COLS,
   DENSE_RENDER_STYLE,
+  READABLE_CHARS_PER_IMAGE,
   renderTextToPngsWithCharLimit,
 } from '../src/core/render.js';
 
-// Default render config: cols=100, 240 lines/img → ~24,000 chars/img if
+// Default render config: cols=100, 90 lines/img → ~9,000 chars/img if
 // lines fully fill the width. For shorter lines, the budget is dominated
 // by row count (each line takes ≥1 row regardless of length).
 const COLS = 100;
-const ROWS_PER_IMG = 240; // floor((1932 - 8) / 8), Spleen 5×8 production cell
+const ROWS_PER_IMG = 90; // floor((728 - 8) / 8), Anthropic-clamped 1568×728 page
 
 describe('estimateImageCount', () => {
   it('returns 1 for empty / tiny text', () => {
@@ -44,13 +45,13 @@ describe('estimateImageCount', () => {
   });
 
   it('scales linearly with row count for short-line content', () => {
-    // Full-canvas policy: 100 cols × 240 rows = 24,000 chars/page.
+    // Full-canvas policy: 100 cols × 90 rows = 9,000 chars/page.
     const oneImage = Array.from({ length: ROWS_PER_IMG }, () => 'x').join('\n');
     expect(estimateImageCount(oneImage, COLS)).toBe(1);
-    // 196 short lines spill into a second page.
+    // ROWS_PER_IMG + 1 short lines spill into a second page.
     const justOver = Array.from({ length: ROWS_PER_IMG + 1 }, () => 'x').join('\n');
     expect(estimateImageCount(justOver, COLS)).toBe(2);
-    // 10 × 240 rows → exactly 10 full pages.
+    // 10 × ROWS_PER_IMG rows → exactly 10 full pages.
     const tenImages = Array.from({ length: ROWS_PER_IMG * 10 }, () => 'x').join('\n');
     expect(estimateImageCount(tenImages, COLS)).toBe(10);
   });
@@ -59,19 +60,19 @@ describe('estimateImageCount', () => {
     // A single 1000-char line wraps to ceil(1000/100) = 10 rows.
     const wrapped = 'x'.repeat(1000);
     expect(estimateImageCount(wrapped, COLS)).toBe(1); // 10 rows, fits in 1 img
-    // 24,000 chars on one line wraps to 240 rows → exactly 1 full page.
-    const oneImg = 'x'.repeat(24_000);
+    // COLS × ROWS_PER_IMG chars on one line wrap to exactly 1 full page.
+    const oneImg = 'x'.repeat(COLS * ROWS_PER_IMG);
     expect(estimateImageCount(oneImg, COLS)).toBe(1);
-    // 24,001 chars overflows into a second page.
-    const twoImgs = 'x'.repeat(24_001);
+    // One char more overflows into a second page.
+    const twoImgs = 'x'.repeat(COLS * ROWS_PER_IMG + 1);
     expect(estimateImageCount(twoImgs, COLS)).toBe(2);
   });
 
   it('also accepts a numeric length (legacy chars-based estimate)', () => {
-    // Numeric path uses the full READABLE_CHARS_PER_IMAGE (50k) budget per page.
+    // Numeric path uses the full READABLE_CHARS_PER_IMAGE budget per page.
     expect(estimateImageCount(0, COLS)).toBe(1);
-    expect(estimateImageCount(50_000, COLS)).toBe(1);
-    expect(estimateImageCount(50_001, COLS)).toBe(2);
+    expect(estimateImageCount(READABLE_CHARS_PER_IMAGE, COLS)).toBe(1);
+    expect(estimateImageCount(READABLE_CHARS_PER_IMAGE + 1, COLS)).toBe(2);
   });
 });
 
@@ -89,10 +90,10 @@ describe('dense readable render profile', () => {
     );
 
     expect(imgs.length).toBeGreaterThanOrEqual(3);
-    // 384 cols × 5 px bare cell + 8 px pad = 1928 px wide — the dense path fills
-    // the ~1932×1932 ceiling (≤2000 px) at the 5×8 cell for Fable / Opus 4.8.
-    expect(imgs[0]!.width).toBeGreaterThan(1900);
-    expect(imgs[0]!.width).toBeLessThanOrEqual(2000);
+    // 312 cols × 5 px bare cell + 8 px pad = 1568 px wide — the dense path fills
+    // Anthropic's long-edge bound exactly (any wider → server-side resample).
+    expect(imgs[0]!.width).toBeGreaterThan(1500);
+    expect(imgs[0]!.width).toBeLessThanOrEqual(1568);
   });
 
   it('pages diff-shaped tool output at the dense readable budget', async () => {
@@ -355,12 +356,14 @@ describe('paging end-to-end (transformRequest)', () => {
   });
 
   it('pages dense medium tool_results instead of packing them into one image', async () => {
-    const lockish = Array.from({ length: 1200 }, (_, i) =>
+    const lockish = Array.from({ length: 600 }, (_, i) =>
       `  pkg-${i}@npm:1.${i}.0(peer@npm:^${i}.0.0)(typescript@npm:^5.${i % 10}.0): checksum=${'a'.repeat(24)}`,
     ).join('\n');
-    // > one dense page (DENSE_CONTENT_CHARS_PER_IMAGE ≈ 92k) so it genuinely pages.
-    expect(lockish.length).toBeGreaterThan(100_000);
-    expect(lockish.length).toBeLessThan(150_000);
+    // > 2 dense pages (DENSE_CONTENT_CHARS_PER_IMAGE = 28,080) so it genuinely
+    // pages, while 600 short rows / 90 per page = 7 images stays under the ≤10
+    // per-tool_result clamp (no truncation).
+    expect(lockish.length).toBeGreaterThan(2 * DENSE_CONTENT_CHARS_PER_IMAGE);
+    expect(lockish.length).toBeLessThan(80_000);
 
     const { info } = await transformRequest(makeReq(lockish), { multiCol: 1, charsPerToken: 2 });
     expect(info.compressed).toBe(true);

tests/render.test.ts

@@ -1943,17 +1943,18 @@ describe('transform', () => {
   // responds to `cols` (which scales chars/image linearly the same way a smaller
   // cell-H would).
 
-  it('maxCharsPerImage: fills the canvas (READABLE_CHARS_PER_IMAGE = 50k)', () => {
-    // Policy: maximum chars per page, full ~1932-px-tall canvas. At cols=100 the
-    // canvas holds 100 × 240 = 24,000 chars per page (height-limited).
-    expect(maxCharsPerImage(100)).toBe(24_000);
+  it('maxCharsPerImage: fills the canvas (READABLE_CHARS_PER_IMAGE = 28,080)', () => {
+    // Policy: maximum chars per page, full 728-px-tall canvas (Anthropic
+    // 1568-edge / ~1.15 MP clamp). At cols=100 the canvas holds
+    // 100 × 90 = 9,000 chars per page (height-limited).
+    expect(maxCharsPerImage(100)).toBe(9_000);
   });
 
-  it('maxCharsPerImage: scales with cols and caps at the 50k page budget', () => {
-    expect(maxCharsPerImage(20)).toBe(4_800);    // 20 × 240 = 4,800 (height-bound)
-    expect(maxCharsPerImage(50)).toBe(12_000);   // 50 × 240 = 12,000 (height-bound)
-    expect(maxCharsPerImage(200)).toBe(48_000);  // 200 × 240 = 48,000 (height-bound)
-    expect(maxCharsPerImage(313)).toBe(50_000);  // 313 × 240 = 75,120 → capped at READABLE
+  it('maxCharsPerImage: scales with cols and caps at the 28k page budget', () => {
+    expect(maxCharsPerImage(20)).toBe(1_800);    // 20 × 90 = 1,800 (height-bound)
+    expect(maxCharsPerImage(50)).toBe(4_500);    // 50 × 90 = 4,500 (height-bound)
+    expect(maxCharsPerImage(200)).toBe(18_000);  // 200 × 90 = 18,000 (height-bound)
+    expect(maxCharsPerImage(313)).toBe(28_080);  // 313 × 90 = 28,170 → capped at READABLE
   });
 
   it('isCompressionProfitable: doubling cols halves the 2-image break-even threshold', () => {