Commit Graph

3 Commits

Author SHA1 Message Date
teamchong e328c93d52 atlas: bit-pack pixel storage (1 bit/pixel) for 3.15× bundle shrink
Unifont glyphs are inherently 1-bit (every pixel is 0 or 255). Storing as
8-bit bytes wasted 7 bits per pixel.

Changes:
- scripts/gen-atlas.ts: emit ATLAS_PIXELS as bit-packed MSB-first. OFFSETS
  now indexes bits not bytes. Per-cell paint thresholds R-channel at 128
  defensively (Unifont is already 0/255).
- src/core/render.ts:blitGlyph: byte read replaced with bit extraction
  (~3 extra ops per pixel; negligible). max() blending removed since bit
  data is binary, no AA edges, no glyph overlap.
- src/core/atlas.ts: regenerated with the new format. Header comment
  documents the bit-extraction formula.
- tests/render.test.ts: +1 structural assertion that ATLAS_PIXELS.byteLength
  matches the bit-packed total.

Bundle size:
  atlas.ts raw:    5.47 MB → 1.06 MB  (5.2× shrink)
  atlas.ts gzip:   1.55 MB → 479 KB   (3.2× shrink)
  dist gzipped:    1.57 MB → 488 KB   (3.15× shrink, well under Workers
                                       free-tier 1 MB cap)

Pre-deflate shrink is 8× (theoretical max). The 3.15× post-deflate
result reflects that deflate was already exploiting long runs of 0/255
in the 8-bit data; the bit-packed bytes are denser and less compressible.

Render output is byte-identical to pre-bit-pack (storage format change
only). 108/108 vitest, 4/4 shell tests, typecheck + build clean.
2026-05-18 21:54:07 -04:00
teamchong f199640bdc ship Unifont @ 10px multilingual rendering + 400 diagnosis fixes
- Switch font: JetBrains Mono @ 15px → Unifont 16.0.04 @ 10px. Sparse
  atlas with double-width EAW dispatch for CJK. Full BMP profile
  (35,501 glyphs) covers Latin/Cyrillic/Greek/Hebrew/Arabic/CJK/Hangul
  plus Letterlike, Misc Technical, Block Elements, Geometric Shapes,
  Misc Symbols, Dingbats, Enclosed Alphanumerics, all box-drawing.
- Fix cell-height bug (was hardcoded 9; now probed from font metrics
  via Math.ceil(maxAscent+maxDescent)). Previously clipped the top
  2 rows off every CJK glyph since the original Unifont ship.
- Expand tabs to spaces with 4-column tab stops, EAW-aware. Was the
  silent-drop source for 99.6% of dropped chars in production.
- Add dropped_codepoints_top telemetry (top 20 per event, only emitted
  when drops > 0) so future coverage gaps surface in events.jsonl.
- Raise minToolResultChars 2000→5000 and minReminderChars 1000→2000
  to stop net-losing tokens on small compressions, per ~2,500 tok/img
  empirical measurement (N=33 cold-miss events).
- Fix dashboard per-image-cost formula: was pngBytes/375 (~190 tok),
  now imageCount * 2500 (the measured empirical rate). Reduces the
  inflated "tokens saved" headline to honest numbers.
- Schema-stub fix in compressSchemas path: preserveSchemaShell()
  retains type / properties / required / enum / oneOf / anyOf / $ref /
  numeric constraints recursively (depth 20), strips only descriptions
  and metadata. Prevents 400 errors on first-tool-call requests.
- Capture full gzipped 4xx body + per-event req_body_sha8 for upstream
  error diagnosis. Sidecar to ~/.pixelpipe/4xx-bodies/ when over 32 KB
  inline cap. Workers-safe via CompressionStream + Web Crypto.
- scripts/restart.sh: pnpm-aware one-command kill+rebuild+start with
  graceful SIGTERM→SIGKILL escalation, build-failure abort, port-in-use
  precheck, --no-build flag for fast iteration. Shell-tested.
- Bundle 1.58 MB gzipped on Node. 91/91 vitest + 4/4 shell tests.
2026-05-18 21:28:25 -04:00
teamchong f91c149d05 Port to TypeScript with dual Node + Cloudflare Workers targets
Runtime is pure Web Standard APIs (fetch, Request, Response, Uint8Array,
CompressionStream, crypto.subtle, btoa) — zero runtime dependencies.

Core (src/core/, runs identically on Node 18+ and Workers):
  - atlas.ts     auto-generated 9x15 JBM glyph atlas, base64-inlined (17KB)
  - png.ts       minimal grayscale PNG encoder via CompressionStream
  - render.ts    text → packed PNG, soft-wraps at 100 cols, ≤1568px tall
  - transform.ts request body rewriter (system + tool docs → image blocks)
  - proxy.ts     fetch-handler that transforms then forwards to Anthropic
  - types.ts     Anthropic Messages API types we touch

Adapters:
  - src/node.ts     node:http server + CLI flag parsing
  - src/worker.ts   export default { fetch } for wrangler dev/deploy

Build tooling:
  - scripts/gen-atlas.ts   @napi-rs/canvas → atlas.ts (build-time only)
  - scripts/build.mjs      esbuild Node bundle; wrangler handles Worker
  - tsconfig.json          strict, ES2022, Workers types

Tests (vitest): 8 passing — PNG signature, base64 round-trip, single +
multi-image renders, transform no-op + compress paths, billing-line strip,
tool fold + stub.

E2E smoke: 16K char system → 2 PNGs (36KB) → mock upstream, 34ms.

Next: replace docs, then verify byte-output parity against legacy/python.
2026-05-18 15:33:23 -04:00