5 Commits

Author SHA1 Message Date
Steven Chong efef8cf0d8 eval: complete Sol and Grok quality runs 2026-07-17 18:29:21 -04:00
Steven Chong 8b61fb2d64 test(eval): add Responses quality matrix 2026-07-11 18:22:46 -04:00
Steven Chong 8b525a1acc feat: Grok packing + IDS, keep Fable-only default, honest chart
Ship Grok 5×8 white/IDS packing and universal IDS/factsheet plumbing while
keeping DEFAULT_MODEL_BASES Fable-only: live pure-image Grok is not
Fable-level; image+factsheet is the working exact path.

Chart shows Grok as a text-window series only (no Grok+pxpipe overlay) and
Fable as the measured pxpipe density point. Add Codex-path quality harnesses
and receipts (multi-seed, fix-matrix, shipped smoke).
2026-07-10 23:50:37 -04:00
Steven Chong cd4c9efddd feat: add model-specific render profiles and recall evidence 2026-07-09 22:25:06 -04:00
Steven Chong 056f970881 test(eval): measure Grok image recall and the factsheet contract
Grok was imaging on the Responses path with no readability data. Add
eval/grok-density: density ladder, climb receipts, and a live
image-vs-factsheet comparison on grok-4.5.

Findings locked into the receipts:
- Production 5x8 image-only confabulates exact IDs (0/4).
- Image + factsheet clears 4/4 exact, 0 confab, gist+guard ok (~70%
  fixture savings).
- Looser pure-image packing can clear exact recall but only ~30%
  savings — not worth the default.

Supports keeping Grok opt-in with dense 5x8 + factsheet when enabled.
2026-07-09 12:44:14 -04:00