Commit Graph

2 Commits

Author SHA1 Message Date
teamchong f199640bdc ship Unifont @ 10px multilingual rendering + 400 diagnosis fixes
- Switch font: JetBrains Mono @ 15px → Unifont 16.0.04 @ 10px. Sparse
  atlas with double-width EAW dispatch for CJK. Full BMP profile
  (35,501 glyphs) covers Latin/Cyrillic/Greek/Hebrew/Arabic/CJK/Hangul
  plus Letterlike, Misc Technical, Block Elements, Geometric Shapes,
  Misc Symbols, Dingbats, Enclosed Alphanumerics, all box-drawing.
- Fix cell-height bug (was hardcoded 9; now probed from font metrics
  via Math.ceil(maxAscent+maxDescent)). Previously clipped the top
  2 rows off every CJK glyph since the original Unifont ship.
- Expand tabs to spaces with 4-column tab stops, EAW-aware. Was the
  silent-drop source for 99.6% of dropped chars in production.
- Add dropped_codepoints_top telemetry (top 20 per event, only emitted
  when drops > 0) so future coverage gaps surface in events.jsonl.
- Raise minToolResultChars 2000→5000 and minReminderChars 1000→2000
  to stop net-losing tokens on small compressions, per ~2,500 tok/img
  empirical measurement (N=33 cold-miss events).
- Fix dashboard per-image-cost formula: was pngBytes/375 (~190 tok),
  now imageCount * 2500 (the measured empirical rate). Reduces the
  inflated "tokens saved" headline to honest numbers.
- Schema-stub fix in compressSchemas path: preserveSchemaShell()
  retains type / properties / required / enum / oneOf / anyOf / $ref /
  numeric constraints recursively (depth 20), strips only descriptions
  and metadata. Prevents 400 errors on first-tool-call requests.
- Capture full gzipped 4xx body + per-event req_body_sha8 for upstream
  error diagnosis. Sidecar to ~/.pixelpipe/4xx-bodies/ when over 32 KB
  inline cap. Workers-safe via CompressionStream + Web Crypto.
- scripts/restart.sh: pnpm-aware one-command kill+rebuild+start with
  graceful SIGTERM→SIGKILL escalation, build-failure abort, port-in-use
  precheck, --no-build flag for fast iteration. Shell-tested.
- Bundle 1.58 MB gzipped on Node. 91/91 vitest + 4/4 shell tests.
2026-05-18 21:28:25 -04:00
teamchong cb8377a6e4 proxy: tee upstream body and extract Anthropic usage tokens
ProxyEvent gains two new fields:
- usage: Anthropic's input/output/cache_creation/cache_read tokens.
- firstByteMs: ms to upstream response headers (durationMs now measures
  end-to-end since the event fires after usage is extracted).

Implementation (teeForUsage):
- Tee the response body. Client gets one side immediately; we read the
  other in the background.
- SSE: scan up to 64 KiB for 'event: message_start' + the data: payload.
  Usage is always in the first event so this terminates within ms of
  upstream first byte.
- JSON: buffer up to 4 MiB and parse.
- Error responses (status >= 400) and bodyless responses skip extraction.
- Drains the tee fully in the background to avoid backpressure.

Without this, the proxy had no way to validate its own value
proposition. Now we can compute cache hit rate, real input-token
savings, etc. directly from log events.

Tests: +3 (JSON usage, SSE message_start usage, error skips extraction).
2026-05-18 16:02:52 -04:00