From 455ab636a2f3778bfcd92a18e839ee9789308759 Mon Sep 17 00:00:00 2001
From: teamchong <25894545+teamchong@users.noreply.github.com>
Date: Sun, 12 Jul 2026 18:59:18 -0400
Subject: [PATCH] eval(patch-probe): billing grid + OCR phase-sweep findings
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Probe 1: both fable-5 and sonnet-5 bill vision on a 28x28 patch grid
(image_tokens = 3 + ceil(W/28)*ceil(H/28)); snap rows≡6 (mod 7),
cols≡4 (mod 28) to avoid stranding paid patch area.

Probe 2: patch-boundary straddling does NOT affect OCR accuracy
(cols: 4.47% vs 4.60%; rows: z=-0.45, 7-offset paired sweep with
line fixed effects). Real misread drivers: high-entropy runs
(bimodal derailment on base64 blobs), 5x8 confusables (w/W, 8/0),
line wraps. Harnesses: count-tokens-sweep, accuracy-phase-probe
(line-DP-aligned scoring), rescore-sweep (offline paired analysis).
---
 eval/patch-probe/FINDINGS.md              |  71 ++++++++
 eval/patch-probe/accuracy-phase-probe.mjs | 192 ++++++++++++++++++++++
 eval/patch-probe/count-tokens-sweep.mjs   |  77 +++++++++
 eval/patch-probe/rescore-sweep.mjs        | 114 +++++++++++++
 eval/patch-probe/results/fable-H-w56.csv  |  87 ++++++++++
 eval/patch-probe/results/fable-W-h56.csv  |  87 ++++++++++
 eval/patch-probe/results/sonnet-H-w56.csv |  87 ++++++++++
 eval/patch-probe/results/sonnet-W-h56.csv |  87 ++++++++++
 8 files changed, 802 insertions(+)
 create mode 100644 eval/patch-probe/FINDINGS.md
 create mode 100644 eval/patch-probe/accuracy-phase-probe.mjs
 create mode 100644 eval/patch-probe/count-tokens-sweep.mjs
 create mode 100644 eval/patch-probe/rescore-sweep.mjs
 create mode 100644 eval/patch-probe/results/fable-H-w56.csv
 create mode 100644 eval/patch-probe/results/fable-W-h56.csv
 create mode 100644 eval/patch-probe/results/sonnet-H-w56.csv
 create mode 100644 eval/patch-probe/results/sonnet-W-h56.csv

diff --git a/eval/patch-probe/FINDINGS.md b/eval/patch-probe/FINDINGS.md
new file mode 100644
index 0000000..9774bfe
--- /dev/null
+++ b/eval/patch-probe/FINDINGS.md
@@ -0,0 +1,71 @@
+# Patch-grid probe findings
+
+## Probe 1 — billing staircase (count_tokens, free) ✅
+
+- Both `claude-fable-5` and `claude-sonnet-5` bill vision on a **28×28 px patch
+  grid**. The "Sonnet uses 32×32" rumor is false: image_tokens steps occur
+  exactly when W or H crosses a multiple of 28, on both axes, both models
+  (CSVs in this dir).
+- Formula: `image_tokens = 3 + ceil(W/28) * ceil(H/28)` (fixed +3 per image).
+  Spot check at H=28: W=53..56 → 5 tokens (2 patches), W=57..60 → 6 (3 patches).
+- The docs' `(W×H)/750` is just an approximation of 784 px²/patch (28²) plus
+  the constant.
+- No server-side resample at ≤1568 px long edge: a 1.73 MP image bills the full
+  grid, so there is no ~1.15 MP cap kicking in — what we render is exactly what
+  the model sees.
+
+## Geometry consequences for pxpipe (CELL 5×8, PAD_X = PAD_Y = 4)
+
+- Height = 8 + 8·rows → hits a 28-multiple iff **rows ≡ 6 (mod 7)**
+  (every 7 rows = 56 px = exactly 2 patch rows).
+  `MAX_HEIGHT_PX = 728` at 90 rows = 26 patch rows, a perfect fit.
+- Width = 8 + 5·cols (+ atlas slack) → hits a 28-multiple iff
+  **cols ≡ 4 (mod 28)** (every 28 cols = 140 px = 5 patch cols).
+  312 cols → 1568 px = 56 patch cols, also a perfect fit at the width cap.
+- Full-cap page: 312 × 90 = 28,080 chars for 3 + 56·26 = **1459 tokens ≈ 19.2
+  chars/token** — the density ceiling for this geometry.
+- Snapping rule: pick rows ≡ 6 (mod 7) and cols ≡ 4 (mod 28); anything else
+  strands already-paid patch area (up to 27 px per axis, ~1–3% of the bill).
+- Phase structure: gcd(5,28)=1 → all 28 horizontal glyph↔patch phases occur in
+  every image; gcd(8,28)=4 → only 7 distinct vertical phases.
+
+## Probe 2 — accuracy vs phase ✅ (verdict: phase alignment does NOT matter)
+
+- Question: do glyphs straddling a patch boundary misread more? A 5 px glyph
+  straddles when `x mod 28 ≥ 24`; an 8 px glyph row straddles when
+  `y mod 28 ≥ 21`. If straddle phases dominate errors, phase-locked pitch
+  (fractional 5.6 / 9.33 px advances, ≈24% density cost) could pay for itself;
+  if flat, keep packed 5×8 and close the alignment theory.
+- Method notes (pitfalls that produced false signals first):
+  - Pure-random char grids trip the safety layer ("looks like credentials") —
+    use real repo source as content.
+  - `temperature` is rejected by fable/sonnet-5 → sampling is stochastic;
+    single runs are NOT repeatable (same image scored 169 vs 237 errs).
+  - Models wrap/merge lines (sonnet emitted 101 lines for 90); positional
+    line pairing turns one slip into a phase-flat ~27% error smear. The
+    harness now does banded line-level DP alignment before char scoring
+    (sonnet went 72.54% → 99.87% on the identical response).
+  - Within one image, row phase aliases content line-type every 7 lines
+    (8·7 = 56 = 2 patches) → row buckets are confounded. Controlled sweep:
+    prepend k = 0..6 blank lines (shifts content by 8k px through all 7 row
+    phases, content identical), then score with per-line fixed effects
+    (`rescore-sweep.mjs`, offline, from dumped responses).
+- **Results (claude-fable-5, 312×90 production geometry):**
+  - Columns (within-image, content-controlled by construction): straddle
+    4.47% vs aligned 4.60% on 3,691 chars — null, per-phase table flat.
+  - Rows (7-offset paired sweep, 439 line×run cells): straddle excess
+    −0.16%±0.28 vs aligned +0.03%±0.29, **z = −0.45** — null.
+  - Real code at full page: fable 99.96% (1/2,276), sonnet 99.87% (3/2,276).
+    28,080 chars for 1,459 image tokens ≈ 19.2 chars/token with ~0.1% CER.
+- **What actually causes misreads** (in error-yield order):
+  1. Long high-entropy runs (base64/hex blobs): bimodal derailment — the
+     same line on the same pixels scored 4% and 73% error across runs. The
+     decoder loses lock mid-run with no language prior to recover; this, not
+     geometry, is the production misread mechanism.
+  2. 5×8 confusables: `w→W` (11× in one page), `s→S`, `c→C`, `K→H`, `M→N`,
+     `8→0`, `(→O`, `:→.` — legibility floor, context-corrected in real code.
+  3. Line wraps on long lines — harmless after alignment, but consumers that
+     trust exact line numbers must re-align.
+- Recommendations: keep packed 5×8 (fractional-advance idea rejected); snap
+  dims per Probe 1 for billing only; route/flag high-entropy lines (e.g.
+  >64 chars of base64-ish content) as literal text instead of pixels.
diff --git a/eval/patch-probe/accuracy-phase-probe.mjs b/eval/patch-probe/accuracy-phase-probe.mjs
new file mode 100644
index 0000000..33ced3d
--- /dev/null
+++ b/eval/patch-probe/accuracy-phase-probe.mjs
@@ -0,0 +1,192 @@
+#!/usr/bin/env node
+// Probe 2: OCR accuracy vs glyph↔patch phase, using the PRODUCTION renderer.
+// gcd(CELL_W=5,28)=1 → one image contains all 28 horizontal phases;
+// gcd(CELL_H=8,28)=4 → 7 vertical phases. Content is gibberish lowercase
+// 4-letter groups (no language prior, not credential-shaped → avoids the
+// refusal classifier that fires on mixed-case alnum strings). Every 5th cell
+// is a space; cols must be a multiple of lcm(5,28)=140 so each phase sees an
+// equal number of letter cells (4 letters + 1 space per phase per 140 cols).
+//
+// Usage: CC_OAUTH_TOKEN=... node accuracy-phase-probe.mjs [model] [cols] [rows] [seed]
+//   cols: multiple of 140, rows: multiple of 7.
+// NOTE: real inference — costs output tokens.
+
+import { writeFile } from 'node:fs/promises';
+import { renderTextToPngs, PAD_X, PAD_Y, CELL_W, CELL_H } from '../../dist/core/render.js';
+
+const model = process.argv[2] ?? 'claude-fable-5';
+const COLS = +(process.argv[3] ?? 140);
+const ROWS = +(process.argv[4] ?? 21);
+const seedArg = process.argv[5] ?? '1';
+const PADL = +(process.argv[6] ?? 0); // blank lines prepended to the IMAGE only: shifts row phase by PADL*CELL_H px, content identical
+const fileMode = !/^\d+$/.test(seedArg); // non-numeric 5th arg = path to a real text file
+const seed = fileMode ? 1 : +seedArg;
+const P = 28;
+if (!fileMode && (COLS % 140 || ROWS % 7)) console.error(`warn: cols%140=${COLS % 140} rows%7=${ROWS % 7} — phases unbalanced`);
+
+let s = (seed >>> 0) || 1;
+const rnd = () => (s ^= s << 13, s ^= s >>> 17, s ^= s << 5, (s >>> 0) / 2 ** 32);
+// Random-order REAL dictionary words: refusal-safe, production-representative.
+// Language prior is phase-independent, so straddle effects survive as relative
+// error-rate differences. Phase balance is statistical, not exact (per-phase
+// totals are tracked, so unevenness is handled in the rates).
+const { readFileSync } = await import('node:fs');
+let grid;
+if (fileMode) {
+  // Production-representative content: real source text, pre-wrapped to COLS.
+  grid = readFileSync(seedArg, 'utf8').split('\n')
+    .map(l => l.replace(/\t/g, '  ').replace(/[^\x20-\x7e]/g, '?').trimEnd().slice(0, COLS))
+    .filter(l => l.trim().length >= 8)
+    .slice(0, ROWS);
+  if (grid.length < ROWS) console.error(`warn: file only yielded ${grid.length} lines`);
+} else {
+  const WORDS = readFileSync('/usr/share/dict/words', 'utf8').split('\n')
+    .filter(w => /^[a-z]{3,7}$/.test(w));
+  grid = Array.from({ length: ROWS }, () => {
+    let line = '';
+    for (;;) {
+      const w = WORDS[(rnd() * WORDS.length) | 0];
+      if (line.length + w.length + (line ? 1 : 0) > COLS) break;
+      line += (line ? ' ' : '') + w;
+    }
+    return line;
+  });
+}
+
+const imgs = await renderTextToPngs('\n'.repeat(PADL) + grid.join('\n'), COLS);
+if (PADL) console.error(`padLines=${PADL} (row shift ${PADL * CELL_H}px, phase +${(PADL * CELL_H) % 28} mod 28)`);
+if (imgs.length !== 1) throw new Error(`expected 1 image, got ${imgs.length}`);
+const img = imgs[0];
+const png = img.png ?? img.data ?? img.buffer ?? Object.values(img).find(v => v instanceof Uint8Array);
+if (!png) throw new Error('no png buffer; keys=' + Object.keys(img).join(','));
+console.error(`image: ${img.width ?? '?'}x${img.height ?? '?'}px ${png.length}B; predicted image_tokens=` +
+  (img.width && img.height ? 3 + Math.ceil(img.width / P) * Math.ceil(img.height / P) : '?'));
+
+const BASE = process.env.ANTHROPIC_BASE_URL ?? 'https://api.anthropic.com';
+const res = await fetch(`${BASE}/v1/messages`, {
+  method: 'POST',
+  headers: {
+    'authorization': `Bearer ${process.env.CC_OAUTH_TOKEN}`,
+    'anthropic-version': '2023-06-01',
+    'anthropic-beta': 'oauth-2025-04-20',
+    'content-type': 'application/json',
+  },
+  body: JSON.stringify({
+    model,
+    max_tokens: ROWS * (COLS + 2) + 1500,
+    system: "You are Claude Code, Anthropic's official CLI for Claude.",
+    messages: [{
+      role: 'user', content: [
+        { type: 'image', source: { type: 'base64', media_type: 'image/png', data: Buffer.from(png).toString('base64') } },
+        { type: 'text', text: `OCR this image to plain text. Preserve line breaks. Output only the text.` },
+      ],
+    }],
+  }),
+});
+const j = await res.json();
+if (!res.ok) { console.error('API error:', JSON.stringify(j)); process.exit(1); }
+await writeFile(`/tmp/phase-probe-${model}-${Date.now()}.json`, JSON.stringify({ model, cols: COLS, rows: ROWS, seedArg, padLines: PADL, grid, resp: j }, null, 2));
+const out = (j.content?.find(b => b.type === 'text')?.text ?? '')
+  .replace(/```[a-z]*\n?/g, '').split('\n').map(l => l.trimEnd()).filter(l => l.length);
+console.error(`usage=${JSON.stringify(j.usage)} stop=${j.stop_reason} lines=${out.length}/${ROWS}`);
+if (out.length < ROWS) console.error('rawTextHead: ' + JSON.stringify((j.content?.find(b => b.type === 'text')?.text ?? '').slice(0, 200)));
+
+// Alignment-based scoring: Levenshtein backtrace marks which truth positions
+// matched exactly; everything else (sub or indel) is an error at its truth pos.
+function alignOk(truth, got, subs) {
+  const T = truth.length, G = got.length;
+  const dp = Array.from({ length: T + 1 }, () => new Array(G + 1).fill(0));
+  for (let i = 0; i <= T; i++) dp[i][0] = i;
+  for (let jj = 0; jj <= G; jj++) dp[0][jj] = jj;
+  for (let i = 1; i <= T; i++) for (let jj = 1; jj <= G; jj++)
+    dp[i][jj] = Math.min(dp[i - 1][jj - 1] + (truth[i - 1] === got[jj - 1] ? 0 : 1), dp[i - 1][jj] + 1, dp[i][jj - 1] + 1);
+  const ok = new Array(T).fill(false);
+  let i = T, jj = G;
+  while (i > 0 && jj > 0) {
+    if (dp[i][jj] === dp[i - 1][jj - 1] + (truth[i - 1] === got[jj - 1] ? 0 : 1)) {
+      if (truth[i - 1] === got[jj - 1]) ok[i - 1] = true;
+      else subs.set(`${truth[i - 1]}->${got[jj - 1]}`, (subs.get(`${truth[i - 1]}->${got[jj - 1]}`) ?? 0) + 1);
+      i--; jj--;
+    } else if (dp[i][jj] === dp[i - 1][jj] + 1) i--;
+    else jj--;
+  }
+  return ok;
+}
+
+// Line-level alignment BEFORE char scoring: the model may emit preamble lines,
+// wrap long lines (1 truth : 2 out), or drop lines. Positional r->r pairing
+// turns one such slip into a phase-flat error smear. Banded DP, merge-aware.
+function lev(a, b) {
+  const m = b.length;
+  let prev = Array.from({ length: m + 1 }, (_, k) => k), cur = new Array(m + 1);
+  for (let i = 1; i <= a.length; i++) {
+    cur[0] = i;
+    for (let k = 1; k <= m; k++)
+      cur[k] = Math.min(prev[k - 1] + (a[i - 1] === b[k - 1] ? 0 : 1), prev[k] + 1, cur[k - 1] + 1);
+    [prev, cur] = [cur, prev];
+  }
+  return prev[m];
+}
+function alignLines(truth, got) {
+  const T = truth.length, G = got.length, BAND = 25, INF = 1e9;
+  const dp = Array.from({ length: T + 1 }, () => new Array(G + 1).fill(INF));
+  const bt = Array.from({ length: T + 1 }, () => new Array(G + 1).fill(null));
+  dp[0][0] = 0;
+  for (let jj = 1; jj <= G; jj++) { dp[0][jj] = dp[0][jj - 1] + 2; bt[0][jj] = ['spur']; }
+  for (let i = 1; i <= T; i++) {
+    dp[i][0] = dp[i - 1][0] + truth[i - 1].length; bt[i][0] = ['miss'];
+    for (let jj = Math.max(1, i - BAND); jj <= Math.min(G, i + BAND); jj++) {
+      let c = dp[i - 1][jj - 1] + lev(truth[i - 1], got[jj - 1]), b = ['m11'];
+      if (jj >= 2 && dp[i - 1][jj - 2] < INF) {
+        const merged = [got[jj - 2] + ' ' + got[jj - 1], got[jj - 2] + got[jj - 1]];
+        for (const mtxt of merged) {
+          const c2 = dp[i - 1][jj - 2] + lev(truth[i - 1], mtxt);
+          if (c2 < c) { c = c2; b = ['m12', mtxt]; }
+        }
+      }
+      if (dp[i][jj - 1] + 2 < c) { c = dp[i][jj - 1] + 2; b = ['spur']; }
+      if (dp[i - 1][jj] + truth[i - 1].length < c) { c = dp[i - 1][jj] + truth[i - 1].length; b = ['miss']; }
+      dp[i][jj] = c; bt[i][jj] = b;
+    }
+  }
+  const matched = new Array(T).fill(null);
+  let i = T, jj = G;
+  while ((i > 0 || jj > 0) && bt[i][jj]) {
+    const b = bt[i][jj];
+    if (b[0] === 'spur') jj--;
+    else if (b[0] === 'miss') i--;
+    else if (b[0] === 'm12') { matched[i - 1] = b[1]; i--; jj -= 2; }
+    else { matched[i - 1] = got[jj - 1]; i--; jj--; }
+  }
+  return matched;
+}
+
+const mk = () => ({ err: new Array(P).fill(0), tot: new Array(P).fill(0) });
+const col = mk(), row = mk();
+let errs = 0, tot = 0, missedLines = 0;
+const subs = new Map();
+const matched = alignLines(grid, out);
+for (let r = 0; r < ROWS && r < grid.length; r++) {
+  const truth = grid[r];
+  if (matched[r] == null) { missedLines++; continue; } // structural, not phase-scorable
+  const ok = alignOk(truth, matched[r], subs);
+  for (let c = 0; c < truth.length; c++) {
+    if (truth[c] === ' ') continue; // letters only
+    const cp = (PAD_X + c * CELL_W) % P, rp = (PAD_Y + (r + PADL) * CELL_H) % P;
+    col.tot[cp]++; row.tot[rp]++; tot++;
+    if (!ok[c]) { errs++; col.err[cp]++; row.err[rp]++; }
+  }
+}
+const agg = (m, pred) => {
+  let e = 0, t = 0;
+  for (let p = 0; p < P; p++) if (m.tot[p] && pred(p)) { e += m.err[p]; t += m.tot[p]; }
+  return t ? `${(100 * e / t).toFixed(2)}% (${e}/${t})` : 'n/a';
+};
+console.log(`model=${model} cols=${COLS} rows=${ROWS} seed=${seed}`);
+console.log(`overall acc=${(100 * (1 - errs / tot)).toFixed(2)}% errs=${errs}/${tot} scoredLines=${grid.length - missedLines}/${ROWS} missedLines=${missedLines} rawOutLines=${out.length}`);
+console.log(`colStraddle(x%28>=24): ${agg(col, p => p >= 24)}  vs aligned: ${agg(col, p => p < 24)}`);
+console.log(`rowStraddle(y%28>=21): ${agg(row, p => p >= 21)}  vs aligned: ${agg(row, p => p < 21)}`);
+console.log('axis,phase,err,tot,errPct');
+for (let p = 0; p < P; p++) if (col.tot[p]) console.log(`col,${p},${col.err[p]},${col.tot[p]},${(100 * col.err[p] / col.tot[p]).toFixed(1)}`);
+for (let p = 0; p < P; p++) if (row.tot[p]) console.log(`row,${p},${row.err[p]},${row.tot[p]},${(100 * row.err[p] / row.tot[p]).toFixed(1)}`);
+console.log('topConfusions: ' + [...subs.entries()].sort((a, b) => b[1] - a[1]).slice(0, 12).map(([k, n]) => `${k}×${n}`).join(' '));
diff --git a/eval/patch-probe/count-tokens-sweep.mjs b/eval/patch-probe/count-tokens-sweep.mjs
new file mode 100644
index 0000000..6631cc3
--- /dev/null
+++ b/eval/patch-probe/count-tokens-sweep.mjs
@@ -0,0 +1,77 @@
+/**
+ * Probe 1: billing-staircase sweep against /v1/messages/count_tokens (free, unbilled).
+ *
+ * Sends blank PNGs of swept dimensions and records input_tokens. If image cost
+ * quantizes as ceil(W/P)*ceil(H/P)*k, step positions in the W (or H) direction
+ * reveal the vision patch size P (28 vs 32 hypothesis). If cost is smooth
+ * ~(W*H)/750, billing is decoupled from the encoder grid and this channel is silent.
+ *
+ * Usage:
+ *   CC_OAUTH_TOKEN=... node count-tokens-sweep.mjs <model> <axis W|H> <fixed> <from> <to>
+ * Output: CSV  w,h,input_tokens,image_tokens  (image_tokens = delta vs no-image baseline)
+ */
+import sharp from 'sharp';
+
+const TOKEN = process.env.CC_OAUTH_TOKEN;
+if (!TOKEN) {
+  console.error('CC_OAUTH_TOKEN not set');
+  process.exit(1);
+}
+
+const API = 'https://api.anthropic.com/v1/messages/count_tokens';
+
+async function count(model, content) {
+  const body = {
+    model,
+    // Constant across all calls -> cancels in the baseline delta.
+    system: [{ type: 'text', text: "You are Claude Code, Anthropic's official CLI for Claude." }],
+    messages: [{ role: 'user', content }],
+  };
+  const res = await fetch(API, {
+    method: 'POST',
+    headers: {
+      'content-type': 'application/json',
+      authorization: `Bearer ${TOKEN}`,
+      'anthropic-version': '2023-06-01',
+      'anthropic-beta': 'oauth-2025-04-20',
+    },
+    body: JSON.stringify(body),
+  });
+  if (!res.ok) throw new Error(`HTTP ${res.status}: ${(await res.text()).slice(0, 300)}`);
+  return (await res.json()).input_tokens;
+}
+
+const blankPng = (w, h) =>
+  sharp({ create: { width: w, height: h, channels: 3, background: { r: 255, g: 255, b: 255 } } })
+    .png()
+    .toBuffer();
+
+const model = process.argv[2] ?? 'claude-fable-5';
+const axis = (process.argv[3] ?? 'W').toUpperCase();
+const fixed = parseInt(process.argv[4] ?? '56', 10);
+const from = parseInt(process.argv[5] ?? '20', 10);
+const to = parseInt(process.argv[6] ?? '100', 10);
+
+const baseline = await count(model, [{ type: 'text', text: 'x' }]);
+console.log(`# model=${model} axis=${axis} fixed=${fixed} baseline=${baseline}`);
+console.log('w,h,input_tokens,image_tokens');
+
+for (let v = from; v <= to; v++) {
+  const [w, h] = axis === 'W' ? [v, fixed] : [fixed, v];
+  const data = (await blankPng(w, h)).toString('base64');
+  let t;
+  for (let attempt = 0; ; attempt++) {
+    try {
+      t = await count(model, [
+        { type: 'image', source: { type: 'base64', media_type: 'image/png', data } },
+        { type: 'text', text: 'x' },
+      ]);
+      break;
+    } catch (e) {
+      if (attempt >= 3) throw e;
+      await new Promise((r) => setTimeout(r, 1000 * (attempt + 1))); // ride out RPM 429s
+    }
+  }
+  console.log(`${w},${h},${t},${t - baseline}`);
+  await new Promise((r) => setTimeout(r, 60));
+}
diff --git a/eval/patch-probe/rescore-sweep.mjs b/eval/patch-probe/rescore-sweep.mjs
new file mode 100644
index 0000000..46af9bd
--- /dev/null
+++ b/eval/patch-probe/rescore-sweep.mjs
@@ -0,0 +1,114 @@
+#!/usr/bin/env node
+// Offline paired re-analysis of the row-phase sweep. Loads dumped responses
+// (/tmp/phase-probe-claude-fable-5-*.json), maps each to its padLines k via the
+// usage token counts in /tmp/phase-sweep-k*.txt, then measures the phase effect
+// with per-line fixed effects: excess(ℓ,k) = errRate(ℓ,k) − mean_k errRate(ℓ,·).
+// Content cancels; only geometry remains. No API calls.
+import { readFileSync, readdirSync } from 'node:fs';
+import { PAD_Y, CELL_H } from '../../dist/core/render.js';
+
+const P = 28;
+
+function lev(a, b) {
+  const m = b.length;
+  let prev = Array.from({ length: m + 1 }, (_, k) => k), cur = new Array(m + 1);
+  for (let i = 1; i <= a.length; i++) {
+    cur[0] = i;
+    for (let k = 1; k <= m; k++)
+      cur[k] = Math.min(prev[k - 1] + (a[i - 1] === b[k - 1] ? 0 : 1), prev[k] + 1, cur[k - 1] + 1);
+    [prev, cur] = [cur, prev];
+  }
+  return prev[m];
+}
+function alignLines(truth, got) {
+  const T = truth.length, G = got.length, BAND = 25, INF = 1e9;
+  const dp = Array.from({ length: T + 1 }, () => new Array(G + 1).fill(INF));
+  const bt = Array.from({ length: T + 1 }, () => new Array(G + 1).fill(null));
+  dp[0][0] = 0;
+  for (let jj = 1; jj <= G; jj++) { dp[0][jj] = dp[0][jj - 1] + 2; bt[0][jj] = ['spur']; }
+  for (let i = 1; i <= T; i++) {
+    dp[i][0] = dp[i - 1][0] + truth[i - 1].length; bt[i][0] = ['miss'];
+    for (let jj = Math.max(1, i - BAND); jj <= Math.min(G, i + BAND); jj++) {
+      let c = dp[i - 1][jj - 1] + lev(truth[i - 1], got[jj - 1]), b = ['m11'];
+      if (jj >= 2 && dp[i - 1][jj - 2] < INF) {
+        for (const mtxt of [got[jj - 2] + ' ' + got[jj - 1], got[jj - 2] + got[jj - 1]]) {
+          const c2 = dp[i - 1][jj - 2] + lev(truth[i - 1], mtxt);
+          if (c2 < c) { c = c2; b = ['m12', mtxt]; }
+        }
+      }
+      if (dp[i][jj - 1] + 2 < c) { c = dp[i][jj - 1] + 2; b = ['spur']; }
+      if (dp[i - 1][jj] + truth[i - 1].length < c) { c = dp[i - 1][jj] + truth[i - 1].length; b = ['miss']; }
+      dp[i][jj] = c; bt[i][jj] = b;
+    }
+  }
+  const matched = new Array(T).fill(null);
+  let i = T, jj = G;
+  while ((i > 0 || jj > 0) && bt[i][jj]) {
+    const b = bt[i][jj];
+    if (b[0] === 'spur') jj--;
+    else if (b[0] === 'miss') i--;
+    else if (b[0] === 'm12') { matched[i - 1] = b[1]; i--; jj -= 2; }
+    else { matched[i - 1] = got[jj - 1]; i--; jj--; }
+  }
+  return matched;
+}
+
+// --- map dumps to k via usage token counts in sweep stdout files ---
+const keyOf = u => `${u.input_tokens}/${u.output_tokens}`;
+const kByKey = new Map();
+for (const f of readdirSync('/tmp').filter(f => /^phase-sweep-k\d\.txt$/.test(f))) {
+  const txt = readFileSync('/tmp/' + f, 'utf8');
+  const m = txt.match(/usage=(\{.*?\}) stop=/s);
+  if (m) kByKey.set(keyOf(JSON.parse(m[1])), +f.match(/k(\d)/)[1]);
+}
+
+const runs = [];
+for (const f of readdirSync('/tmp').filter(f => f.startsWith('phase-probe-claude-fable-5-') && f.endsWith('.json'))) {
+  const d = JSON.parse(readFileSync('/tmp/' + f, 'utf8'));
+  if (!(d.seedArg ?? '').includes('atlas')) continue;
+  const k = kByKey.get(keyOf(d.resp.usage));
+  runs.push({ file: f, k: k ?? 0, replicate: k == null, grid: d.grid, resp: d.resp });
+}
+console.log(`runs: ${runs.map(r => `k=${r.k}${r.replicate ? '(rep)' : ''}`).join(' ')}`);
+
+// --- per-line error rates ---
+const L = runs[0].grid.length;
+const cells = []; // {l, k, rate, phase, straddle}
+for (const run of runs) {
+  const out = (run.resp.content?.find(b => b.type === 'text')?.text ?? '')
+    .replace(/```[a-z]*\n?/g, '').split('\n').map(l => l.trimEnd()).filter(l => l.length);
+  const matched = alignLines(run.grid, out);
+  for (let l = 0; l < L; l++) {
+    if (matched[l] == null) continue;
+    const phase = (PAD_Y + (l + run.k) * CELL_H) % P;
+    cells.push({ l, k: run.k, rate: lev(run.grid[l], matched[l]) / run.grid[l].length, phase, straddle: phase >= 21 });
+  }
+}
+
+// per-line means (fixed effect)
+const byLine = new Map();
+for (const c of cells) (byLine.get(c.l) ?? byLine.set(c.l, []).get(c.l)).push(c.rate);
+const lineMean = new Map([...byLine].map(([l, rs]) => [l, rs.reduce((a, b) => a + b, 0) / rs.length]));
+for (const c of cells) c.excess = c.rate - lineMean.get(c.l);
+
+const bucket = (sel) => {
+  const xs = cells.filter(sel).map(c => c.excess);
+  const n = xs.length, mean = xs.reduce((a, b) => a + b, 0) / n;
+  const sd = Math.sqrt(xs.reduce((a, b) => a + (b - mean) ** 2, 0) / (n - 1));
+  return { n, mean, se: sd / Math.sqrt(n) };
+};
+console.log('\nphase, n(line×run), excessErr%, ±SE%');
+for (const p of [...new Set(cells.map(c => c.phase))].sort((a, b) => a - b)) {
+  const { n, mean, se } = bucket(c => c.phase === p);
+  console.log(`${String(p).padStart(2)}${p >= 21 ? '*' : ' '} , ${n}, ${(100 * mean).toFixed(2)}, ±${(100 * se).toFixed(2)}`);
+}
+const s = bucket(c => c.straddle), a = bucket(c => !c.straddle);
+console.log(`\nstraddle(≥21): ${(100 * s.mean).toFixed(2)}%±${(100 * s.se).toFixed(2)} (n=${s.n})  aligned: ${(100 * a.mean).toFixed(2)}%±${(100 * a.se).toFixed(2)} (n=${a.n})  z=${((s.mean - a.mean) / Math.hypot(s.se, a.se)).toFixed(2)}`);
+
+// hardest lines: err across k to eyeball content-vs-geometry
+const hard = [...lineMean].sort((x, y) => y[1] - x[1]).slice(0, 6);
+console.log('\nhardest lines (idx, meanErr%, per-run rate% by k, head):');
+for (const [l, m] of hard) {
+  const per = cells.filter(c => c.l === l).sort((x, y) => x.k - y.k).map(c => `k${c.k}${c.straddle ? '*' : ''}:${(100 * c.rate).toFixed(0)}`);
+  console.log(`#${l} ${(100 * m).toFixed(1)}% [${per.join(' ')}] ${JSON.stringify(runs[0].grid[l].slice(0, 48))}`);
+}
diff --git a/eval/patch-probe/results/fable-H-w56.csv b/eval/patch-probe/results/fable-H-w56.csv
new file mode 100644
index 0000000..e2b0807
--- /dev/null
+++ b/eval/patch-probe/results/fable-H-w56.csv
@@ -0,0 +1,87 @@
+# model=claude-fable-5 axis=H fixed=56 baseline=31
+w,h,input_tokens,image_tokens
+56,16,36,5
+56,17,36,5
+56,18,36,5
+56,19,36,5
+56,20,36,5
+56,21,36,5
+56,22,36,5
+56,23,36,5
+56,24,36,5
+56,25,36,5
+56,26,36,5
+56,27,36,5
+56,28,36,5
+56,29,38,7
+56,30,38,7
+56,31,38,7
+56,32,38,7
+56,33,38,7
+56,34,38,7
+56,35,38,7
+56,36,38,7
+56,37,38,7
+56,38,38,7
+56,39,38,7
+56,40,38,7
+56,41,38,7
+56,42,38,7
+56,43,38,7
+56,44,38,7
+56,45,38,7
+56,46,38,7
+56,47,38,7
+56,48,38,7
+56,49,38,7
+56,50,38,7
+56,51,38,7
+56,52,38,7
+56,53,38,7
+56,54,38,7
+56,55,38,7
+56,56,38,7
+56,57,40,9
+56,58,40,9
+56,59,40,9
+56,60,40,9
+56,61,40,9
+56,62,40,9
+56,63,40,9
+56,64,40,9
+56,65,40,9
+56,66,40,9
+56,67,40,9
+56,68,40,9
+56,69,40,9
+56,70,40,9
+56,71,40,9
+56,72,40,9
+56,73,40,9
+56,74,40,9
+56,75,40,9
+56,76,40,9
+56,77,40,9
+56,78,40,9
+56,79,40,9
+56,80,40,9
+56,81,40,9
+56,82,40,9
+56,83,40,9
+56,84,40,9
+56,85,42,11
+56,86,42,11
+56,87,42,11
+56,88,42,11
+56,89,42,11
+56,90,42,11
+56,91,42,11
+56,92,42,11
+56,93,42,11
+56,94,42,11
+56,95,42,11
+56,96,42,11
+56,97,42,11
+56,98,42,11
+56,99,42,11
+56,100,42,11
diff --git a/eval/patch-probe/results/fable-W-h56.csv b/eval/patch-probe/results/fable-W-h56.csv
new file mode 100644
index 0000000..650451c
--- /dev/null
+++ b/eval/patch-probe/results/fable-W-h56.csv
@@ -0,0 +1,87 @@
+# model=claude-fable-5 axis=W fixed=56 baseline=31
+w,h,input_tokens,image_tokens
+16,56,36,5
+17,56,36,5
+18,56,36,5
+19,56,36,5
+20,56,36,5
+21,56,36,5
+22,56,36,5
+23,56,36,5
+24,56,36,5
+25,56,36,5
+26,56,36,5
+27,56,36,5
+28,56,36,5
+29,56,38,7
+30,56,38,7
+31,56,38,7
+32,56,38,7
+33,56,38,7
+34,56,38,7
+35,56,38,7
+36,56,38,7
+37,56,38,7
+38,56,38,7
+39,56,38,7
+40,56,38,7
+41,56,38,7
+42,56,38,7
+43,56,38,7
+44,56,38,7
+45,56,38,7
+46,56,38,7
+47,56,38,7
+48,56,38,7
+49,56,38,7
+50,56,38,7
+51,56,38,7
+52,56,38,7
+53,56,38,7
+54,56,38,7
+55,56,38,7
+56,56,38,7
+57,56,40,9
+58,56,40,9
+59,56,40,9
+60,56,40,9
+61,56,40,9
+62,56,40,9
+63,56,40,9
+64,56,40,9
+65,56,40,9
+66,56,40,9
+67,56,40,9
+68,56,40,9
+69,56,40,9
+70,56,40,9
+71,56,40,9
+72,56,40,9
+73,56,40,9
+74,56,40,9
+75,56,40,9
+76,56,40,9
+77,56,40,9
+78,56,40,9
+79,56,40,9
+80,56,40,9
+81,56,40,9
+82,56,40,9
+83,56,40,9
+84,56,40,9
+85,56,42,11
+86,56,42,11
+87,56,42,11
+88,56,42,11
+89,56,42,11
+90,56,42,11
+91,56,42,11
+92,56,42,11
+93,56,42,11
+94,56,42,11
+95,56,42,11
+96,56,42,11
+97,56,42,11
+98,56,42,11
+99,56,42,11
+100,56,42,11
diff --git a/eval/patch-probe/results/sonnet-H-w56.csv b/eval/patch-probe/results/sonnet-H-w56.csv
new file mode 100644
index 0000000..a23b336
--- /dev/null
+++ b/eval/patch-probe/results/sonnet-H-w56.csv
@@ -0,0 +1,87 @@
+# model=claude-sonnet-5 axis=H fixed=56 baseline=31
+w,h,input_tokens,image_tokens
+56,16,36,5
+56,17,36,5
+56,18,36,5
+56,19,36,5
+56,20,36,5
+56,21,36,5
+56,22,36,5
+56,23,36,5
+56,24,36,5
+56,25,36,5
+56,26,36,5
+56,27,36,5
+56,28,36,5
+56,29,38,7
+56,30,38,7
+56,31,38,7
+56,32,38,7
+56,33,38,7
+56,34,38,7
+56,35,38,7
+56,36,38,7
+56,37,38,7
+56,38,38,7
+56,39,38,7
+56,40,38,7
+56,41,38,7
+56,42,38,7
+56,43,38,7
+56,44,38,7
+56,45,38,7
+56,46,38,7
+56,47,38,7
+56,48,38,7
+56,49,38,7
+56,50,38,7
+56,51,38,7
+56,52,38,7
+56,53,38,7
+56,54,38,7
+56,55,38,7
+56,56,38,7
+56,57,40,9
+56,58,40,9
+56,59,40,9
+56,60,40,9
+56,61,40,9
+56,62,40,9
+56,63,40,9
+56,64,40,9
+56,65,40,9
+56,66,40,9
+56,67,40,9
+56,68,40,9
+56,69,40,9
+56,70,40,9
+56,71,40,9
+56,72,40,9
+56,73,40,9
+56,74,40,9
+56,75,40,9
+56,76,40,9
+56,77,40,9
+56,78,40,9
+56,79,40,9
+56,80,40,9
+56,81,40,9
+56,82,40,9
+56,83,40,9
+56,84,40,9
+56,85,42,11
+56,86,42,11
+56,87,42,11
+56,88,42,11
+56,89,42,11
+56,90,42,11
+56,91,42,11
+56,92,42,11
+56,93,42,11
+56,94,42,11
+56,95,42,11
+56,96,42,11
+56,97,42,11
+56,98,42,11
+56,99,42,11
+56,100,42,11
diff --git a/eval/patch-probe/results/sonnet-W-h56.csv b/eval/patch-probe/results/sonnet-W-h56.csv
new file mode 100644
index 0000000..75c4299
--- /dev/null
+++ b/eval/patch-probe/results/sonnet-W-h56.csv
@@ -0,0 +1,87 @@
+# model=claude-sonnet-5 axis=W fixed=56 baseline=31
+w,h,input_tokens,image_tokens
+16,56,36,5
+17,56,36,5
+18,56,36,5
+19,56,36,5
+20,56,36,5
+21,56,36,5
+22,56,36,5
+23,56,36,5
+24,56,36,5
+25,56,36,5
+26,56,36,5
+27,56,36,5
+28,56,36,5
+29,56,38,7
+30,56,38,7
+31,56,38,7
+32,56,38,7
+33,56,38,7
+34,56,38,7
+35,56,38,7
+36,56,38,7
+37,56,38,7
+38,56,38,7
+39,56,38,7
+40,56,38,7
+41,56,38,7
+42,56,38,7
+43,56,38,7
+44,56,38,7
+45,56,38,7
+46,56,38,7
+47,56,38,7
+48,56,38,7
+49,56,38,7
+50,56,38,7
+51,56,38,7
+52,56,38,7
+53,56,38,7
+54,56,38,7
+55,56,38,7
+56,56,38,7
+57,56,40,9
+58,56,40,9
+59,56,40,9
+60,56,40,9
+61,56,40,9
+62,56,40,9
+63,56,40,9
+64,56,40,9
+65,56,40,9
+66,56,40,9
+67,56,40,9
+68,56,40,9
+69,56,40,9
+70,56,40,9
+71,56,40,9
+72,56,40,9
+73,56,40,9
+74,56,40,9
+75,56,40,9
+76,56,40,9
+77,56,40,9
+78,56,40,9
+79,56,40,9
+80,56,40,9
+81,56,40,9
+82,56,40,9
+83,56,40,9
+84,56,40,9
+85,56,42,11
+86,56,42,11
+87,56,42,11
+88,56,42,11
+89,56,42,11
+90,56,42,11
+91,56,42,11
+92,56,42,11
+93,56,42,11
+94,56,42,11
+95,56,42,11
+96,56,42,11
+97,56,42,11
+98,56,42,11
+99,56,42,11
+100,56,42,11