Compare commits

..

35 Commits

Author SHA1 Message Date
sdwolf4103 d700f4877f Merge pull request #4 from sdwolf4103/feat/memory-quality-cleanup
Release v1.4.0 memory quality cleanup
2026-04-28 14:53:12 +08:00
Ralph Chang c0a083ddaf fix(memory): isolate test workspace cleanup 2026-04-28 14:50:30 +08:00
Ralph Chang 8e07bfe3c1 fix(memory): address quality cleanup audit findings 2026-04-28 14:29:28 +08:00
Ralph Chang c7088a8a6e docs(memory): document conservative quality cleanup migration 2026-04-28 14:19:18 +08:00
Ralph Chang efed9e5585 test(memory): add real workspace quality cleanup regression fixture 2026-04-28 14:17:43 +08:00
Ralph Chang 7de10c5808 feat(memory): add local quality cleanup audit logs 2026-04-28 14:17:17 +08:00
Ralph Chang 12eddc2f8c fix(memory): make quality cleanup migration conservative 2026-04-28 14:15:34 +08:00
Ralph Chang 5e85d098d8 chore: prepare v1.4.0 release 2026-04-28 13:37:14 +08:00
Ralph Chang 99c6b97c96 fix: unify all memory quality rules in single module 2026-04-28 13:34:33 +08:00
Ralph Chang 83dcfb479c fix: auto-supersede low-quality compaction memories 2026-04-28 13:29:28 +08:00
Ralph Chang ed6005f6cf fix: tighten compaction memory candidate prompt 2026-04-28 13:24:43 +08:00
Ralph Chang 069ec8ecbb fix: unify workspace memory quality gate 2026-04-28 13:21:15 +08:00
Ralph Chang 60c7019820 chore: prepare v1.3.3 release 2026-04-28 13:06:14 +08:00
Ralph Chang 1847f63480 fix: owner scope in global unowned promotion
Problem: clearPendingMemories() and recordPromotionRejections() would
incorrectly clear or mutate owned entries during global unowned promotion.

Fixes:
1. clearPendingMemories() now respects owner/unowned scope:
   - global clearUnowned only clears unowned same-key entries
   - owned same-key entries are preserved
   - explicit global clear-all-by-key fallback still works

2. recordPromotionRejections() now has includeUnownedOnly option:
   - global unowned rejection only increments/exhausts unowned entries
   - owned same-key entries are preserved

3. Added regression tests:
   - global unowned clear keeps owned same-key entries
   - global unowned rejection only exhausts unowned same-key entries

Tests: 182 pass, 0 fail
2026-04-28 12:27:46 +08:00
Ralph Chang 8b21325469 fix: cross-process lock stale judgment and heartbeat
Problem: CI test "updateJSON serializes writes across separate node processes"
was failing with expect 100 but got 89/97. The root cause was isLockStale()
being too aggressive - it could mistakenly delete locks held by other processes.

Fixes:
1. isLockStale() now uses mtime only - fresh locks are never stale
2. Added heartbeat mechanism during lock hold to support long updaters
3. Removed PID check that was unreliable in CI/containers
4. Fixed ENOENT race when lock is released between EEXIST and stat

Tests: 180 pass, 0 fail
2026-04-28 12:24:56 +08:00
Ralph Chang b846b34e30 feat: implement Plan 1 - Critical Stability fixes
Wave 1: Storage and Journal Safety
- Add frozen cache TTL (1h) and size bounds (50 sessions)
- Add pending journal source-aware retention (compaction-only TTL)
- Add inter-process file lock with stale recovery
- Move processLatestUserMessage to first transform (after isSubAgent guard)

Wave 2: Promotion Ownership and Bounded Rejection
- Add pendingOwnerSessionID/pendingMessageID metadata
- Add owner-aware pending journal clearing
- Add explicit/manual bounded retry (max 3 attempts)
- Fix session.deleted cleanup idempotency

Wave 3: Normalize, Security, and Cache Hardening
- Fix load-time write loop (only write on security/migration change)
- Add deterministic sort tie-breaker (createdAt -> id)
- Add Bearer token redaction
- Add processed message cache bounds
- Remove priorityWithFreshness dead code

Tests: 180 pass, 0 fail
2026-04-28 11:59:29 +08:00
Ralph Chang 47905921ca fix: run compatibility CI on Node 24 2026-04-27 22:13:23 +08:00
Ralph Chang ca88193f9f fix: support CI installs without lockfile 2026-04-27 22:04:11 +08:00
Ralph Chang 1927cc8828 chore: prepare v1.3.1 release 2026-04-27 22:00:04 +08:00
Ralph Chang 64f86ef39c refactor: make memory dedupe repo-agnostic 2026-04-27 21:19:42 +08:00
Ralph Chang 39d27e8d3c docs: note PR 3 security hardening 2026-04-27 20:22:26 +08:00
Ralph Chang 77bf8af3fe test: cover security hardening edge cases 2026-04-27 20:22:09 +08:00
Ralph Chang 6eb341f43c merge: integrate PR #3 security hardening 2026-04-27 20:14:08 +08:00
Ralph Chang 6a1fa525dc docs: document concise compatibility limitations 2026-04-27 19:57:21 +08:00
Ralph Chang d6875aac1b fix: cap and prune pending memory journal 2026-04-27 18:54:44 +08:00
Ralph Chang c2ee245620 test: add opencode plugin compatibility checks 2026-04-27 18:54:14 +08:00
Steven Choo 15c0c8a45d feat: implement indirect prompt injection protection and expanded secret redaction 2026-04-27 12:42:20 +02:00
Ralph Chang 909fec9abb docs: prepare v1.3.0 release notes 2026-04-27 17:06:43 +08:00
Ralph Chang ef1248f23a feat: add consolidation accounting for workspace memory promotion
P0 implementation with four waves:

Wave 1: Dedup with accounting
- Add dedupeLongTermEntriesWithAccounting()
- Classify exact duplicate, identity duplicate, topic duplicate

Wave 2: Normalization with accounting
- Add normalizeWorkspaceMemoryWithAccounting()
- Chain redaction → migration → enforceLongTermLimitsWithAccounting

Wave 3: Promotion accounting integration
- Update accountPendingPromotions() to use new accounting API
- Add supersededKeys to classification
- Distinguish promoted / absorbed / superseded / rejected

Wave 4: Integration tests
- End-to-end tests covering full pipeline

Bug fixes:
- Fix active vs superseded boundary (superseded entries no longer block promotion)
- Remove unused rejected_duplicate_lower_quality type
- Defer pending journal safety cap (TODO added)

Tests: 135 passing (up from 115)
2026-04-27 16:45:55 +08:00
Ralph Chang c8c7dbed3b chore: ignore superpowers plans and update architecture doc
- Add docs/superpowers/plans/ to .gitignore
- Remove tracked plan files from git
- Update docs/architecture.md:
  - Change primary extraction format from XML to 'Memory candidates:'
  - Mark XML format as legacy/deprecated
  - Fix hot session state injection example
2026-04-27 14:53:07 +08:00
Ralph Chang bfa2972353 feat: sharpen compaction memory extraction prompt
Wave 3 of memory quality optimization plan.

- Add good memory examples in buildCompactionPrompt()
- Add bad memory examples to skip (test counts, commit hashes, etc.)
- Add prompt assertions in tests to prevent regression
- Emphasize 'useful if a new agent opens this workspace next week'
2026-04-27 14:40:32 +08:00
Ralph Chang 5fe4955057 test: add memory quality eval fixtures
Wave 2 of memory quality optimization plan.

- 5 accepted cases: durable facts that should be kept
- 7 rejected cases: noise that should be filtered
- Parser-level regression guard (zero API call)
- All cases pass against current extractors.ts
2026-04-27 14:34:53 +08:00
Ralph Chang 55e163adef fix: account for absorbed pending memories
- Add workspaceMemoryIdentityKey() to unify dedup/supersession identity semantics
- Add accountPendingPromotions() to distinguish promoted/absorbed/rejected
- Wire promotion accounting into promotePendingMemories()
- Add clearableKeys.size > 0 guard to prevent journal wipe
- Add regression tests for absorbed duplicate, cap-rejected, all-rejected edge cases

Wave 1 of memory quality optimization plan.
2026-04-27 14:27:43 +08:00
Ralph Chang 5ed57943d2 docs: add memory quality optimization implementation plan
P0 implementation plan with 3 waves:
- Wave 1: Promotion accounting (fix absorbed duplicate data loss)
- Wave 2: Memory quality eval (fixture-based regression guard)
- Wave 3: Compaction prompt negative examples

Addresses architecture review feedback:
- Distinguish promoted/absorbed/rejected pending memories
- Add clearableKeys.size > 0 guard to prevent journal wipe
- Add regression tests for all edge cases
2026-04-27 14:16:46 +08:00
sdwolf4103 2fc2172d59 Fix formatting in README.md 2026-04-27 13:00:26 +08:00
62 changed files with 383 additions and 7880 deletions
-2
View File
@@ -51,8 +51,6 @@ pnpm-lock.yaml
# Superpowers local planning artifacts
docs/superpowers/plans/
docs/plans/
docs/disruptions/
# Local dev/admin script inputs
scripts/dev/run-migration-roots.local.txt
+7 -7
View File
@@ -111,7 +111,7 @@ export type LongTermSource = "explicit" | "compaction" | "manual";
// ✅ USE: const assertions for limits
export const LONG_TERM_LIMITS = {
maxRenderedChars: 3600,
maxRenderedChars: 5200,
maxEntries: 28,
} as const;
```
@@ -140,8 +140,8 @@ const maxEntries = 28;
async function loadWorkspaceMemory() { }
// ✅ REQUIRED: SCREAMING_SNAKE_CASE for constants
const LONG_TERM_LIMITS = { maxRenderedChars: 3600, maxEntries: 28 };
const HOT_STATE_LIMITS = { maxRenderedChars: 700 };
const LONG_TERM_LIMITS = { maxRenderedChars: 5200, maxEntries: 28 };
const HOT_STATE_LIMITS = { maxRenderedChars: 1200 };
// ✅ REQUIRED: PascalCase for types
type WorkspaceMemoryStore = { ... };
@@ -236,7 +236,7 @@ export default {
- **Location**: `~/.local/share/opencode-working-memory/workspaces/{workspaceKey}/workspace-memory.json`
- **Workspace Key**: First 16 chars of `sha256(realpath(workspaceRoot))`
- **Schema**: See `src/types.ts:WorkspaceMemoryStore`
- **Limits**: 3600 chars, 28 entries max
- **Limits**: 5200 chars, 28 entries max
### Session State Files
@@ -299,9 +299,9 @@ Extracts workspace memory candidates from conversation, applies quality gate and
## Performance Considerations
- **Workspace memory budget**: 3600 chars injected into system prompt
- **Session state budget**: 700 chars injected into system prompt
- **Total overhead**: typically well below configured maximums
- **Workspace memory budget**: 5200 chars injected into system prompt
- **Session state budget**: 1200 chars injected into system prompt
- **Total overhead**: ~1500-6000 chars per message (minimal)
- **Storage footprint**: ~2-5 KB per workspace for memory, ~1-3 KB per session
## Contributing
-80
View File
@@ -5,86 +5,6 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.5.4] - 2026-05-02
### Changed
- Centralized the supported `memory-diag` command metadata for the official public commands: `status`, `rejected`, `missing`, and `explain`.
- Removed pre-public legacy aliases so unsupported spellings now use the standard unknown-subcommand path instead of carrying v2.0 compatibility debt.
- Kept `coverage` and `audit` as hidden maintainer diagnostics with neutral maintainer-only notices.
- Removed public npm script exposure for the internal workspace cleanup tool while keeping the development tool in the repository.
### Fixed
- Moved missing-memory disappearance detail formatting into the official `missing` formatter before deleting legacy disappearance formatter code.
## [1.5.3] - 2026-05-02
### Added
- Published the read-only `memory-diag` package binary for package-qualified `npx --package opencode-working-memory memory-diag` usage.
- Added user-facing diagnostics commands for concise status, rejected memory review, missing memory review, and per-memory explanation.
### Changed
- Reworked the diagnostics CLI into focused command, model, formatter, and utility modules while preserving legacy aliases and maintainer diagnostics.
- Updated README and configuration docs to document the supported `memory-diag` CLI workflow and package-qualified `npx` usage.
- Raised the Node.js requirement to `>=22.6.0` for the TypeScript diagnostics CLI runtime.
### Fixed
- Avoided repeated workspace snapshot construction in status diagnostics.
- Replaced per-memory evidence summary lookups with grouped evidence processing to avoid N+1 diagnostics work.
- Added top-level CLI error handling for cleaner user-facing failures.
- Removed a diagnostics model circular dependency by extracting evidence grouping helpers to a neutral module.
## [1.5.1] - 2026-04-30
### Added
- Per-workspace evidence log for extraction, promotion, reinforcement, render, storage, and hook lifecycle events.
- `memory-diag health --json` for machine-readable diagnostics.
- `memory-diag explain` for per-memory render status, strength, reasons, and evidence event IDs.
- `memory-diag trace --memory <id>` for memory lifecycle history.
- UTC calendar-day reinforcement gate so repeated matches cannot inflate a memory multiple times in the same day.
### Changed
- Retention constants and calculations moved to `src/retention.ts`.
- `safetyCritical` is now fully inert: no retention multiplier and no type-cap bypass, while remaining JSON-compatible.
## [1.5.0] - 2026-04-29
### Added
- Strength-based workspace memory retention using exponential decay instead of additive priority scoring.
- Per-type rendered caps for workspace memory candidates: feedback 10, decision 10, project 8, and reference 6.
- Dormant-workspace effective age: after 14 days without activity, additional dormant time counts at 0.25x for retention decay.
- Reinforcement tracking for repeated memories, with same-session and one-hour guards to prevent accidental reinforcement spam.
- Memory health diagnostics for stored vs rendered counts, type caps, global cap overflow, dormancy, retention monitoring, and strength-ranked top/weakest entries.
- CLI smoke tests and regression fixtures covering retention decay, stale-prune removal, type caps, reinforcement, invalid timestamps, and diagnostics.
### Changed
- Workspace memory rendering now ranks entries by retention strength, not the previous priority/penalty model.
- Confidence is retained for compatibility but no longer affects retention scoring.
- Deprecated `safetyCritical` is retained for JSON compatibility but no longer affects retention strength or type-cap behavior.
- Old or stale-marked memories are no longer hard-pruned; they remain stored and only fall out of rendered context through strength and cap competition.
- Existing duplicate promotion and dedupe paths now reinforce the surviving memory instead of only absorbing the duplicate.
- Health output now separates stored active memories from rendered candidates to make cap behavior easier to understand.
- Default prompt budgets are lower after calibration against observed rendered output: workspace memory is 3600 characters and hot session state is 700 characters.
### Fixed
- Invalid `updatedAt` or `retentionClock` values no longer produce `NaN` retention strength or unstable sorting.
- Dormant age calculation only discounts the dormant overlap since an entry was created, so new memories do not inherit old workspace dormancy.
- Type max totals above the global cap are handled correctly: the global rendered limit still wins.
### Not Included Yet
- Delete tombstones and explicit `supersedes` chain enforcement remain deferred follow-up work.
- Hot/warm/cold tiered storage remains a future v1.6 direction.
## [1.4.0] - 2026-04-28
### Added
+16 -60
View File
@@ -5,9 +5,9 @@
Automatic memory for OpenCode agents.
Working memory is context that **remembers what matters, fades what changes, and stays out of the way.**
OpenCode Working Memory helps your agent keep useful context across compactions and sessions: project decisions, preferences, important references, active files, and unresolved errors.
OpenCode Working Memory preserves project decisions, preferences, and references across compactions and sessions, while keeping active files and unresolved errors fresh for the current session — with no manual tools or extra LLM/API calls.
It works automatically, without manual memory tools or extra LLM/API calls.
## Why This Exists
@@ -31,7 +31,6 @@ Use it when you want your agent to remember things like:
- **Compaction-based extraction** — memory extraction piggybacks on OpenCode’s existing compaction flow.
- **No manual tools** — memory is injected automatically into the system prompt.
- **Quality guards** — filters noisy memories, temporary progress snapshots, stack traces, raw errors, and credentials.
- **Retention decay** — keeps the strongest memories in prompt context while older or weaker memories fade out naturally; important and reinforced memories decay more slowly.
## Installation
@@ -122,27 +121,6 @@ Memory types:
- `decision` — important implementation or architecture decisions
- `reference` — useful paths, commands, or configuration references
### Retention Decay
> **Memory should fade, so the agent can keep learning.**
>
> Important memories decay more slowly, but every memory must leave room for newer project reality.
Memories decay over time. The strongest stay visible in the prompt; weaker ones fade from context without being deleted.
```text
strength
│
██ │╲____ reinforced: slower decline
│ ╲______
▒▒ │ ╲__ ordinary memory
│ ╲
├ ─ ─ ─ ─ ─ ─ ─ ─╲─ dynamic cap competition zone
░░ │ ╲ easier for new memories to replace
│ ↑ still stored, not deleted
└──────────────────────────────→ time / sessions
```
## Explicit Memory Triggers
You can explicitly ask the agent to remember durable facts.
@@ -189,46 +167,23 @@ It includes guards for:
- Credential redaction
- Duplicate memory cleanup
- Accounting for promoted, absorbed, superseded, and rejected memories
- Strength-based retention so useful memories stay visible without hard age pruning
- Superseding older decisions with newer ones
- Consolidation accounting so promoted, absorbed, superseded, and rejected memories are handled differently
- Filtering stack traces, git hashes, raw errors, and noisy path-heavy facts
- Rejecting temporary project progress snapshots
The goal is to remember durable facts, not every detail.
**Good memory is selective memory.**
Historical cleanup is intentionally conservative: extraction-time filtering may reject more aggressively, but one-time migration cleanup only supersedes high-confidence garbage patterns. This protects existing durable memories written in declarative style, such as "API endpoint is X" or "Product branding is Y".
### Memory Diagnostics CLI
Use the read-only diagnostics CLI when you want to understand what memory is doing for the current workspace.
| Question | Command |
|---|---|
| Is memory healthy? | `npx --package opencode-working-memory memory-diag` or `npx --package opencode-working-memory memory-diag status` |
| Why was something rejected? | `npx --package opencode-working-memory memory-diag rejected` |
| Where did my memory go? | `npx --package opencode-working-memory memory-diag missing` |
| Why is this memory shown or hidden? | `npx --package opencode-working-memory memory-diag explain <memory-id>` |
Global options:
- `--workspace <path>` — inspect another workspace; defaults to the current directory.
- `--verbose` — show detailed diagnostics.
- `--json` — print machine-readable output where supported.
Examples:
For local development cleanup, use:
```bash
npx --package opencode-working-memory memory-diag status
npx --package opencode-working-memory memory-diag rejected --verbose
npx --package opencode-working-memory memory-diag missing --workspace /path/to/project
npx --package opencode-working-memory memory-diag status --json
npm run cleanup:workspaces -- --dry-run
npm run cleanup:workspaces -- --quarantine
```
The npm package is opencode-working-memory; the installed bin is memory-diag, so package-qualified npx avoids resolving a different package named memory-diag.
Maintainer-only diagnostics and cleanup commands are intentionally not documented here. Future work: move those internal commands to `docs/development.md`.
The cleanup command only quarantines definite temp/test workspace residues by default. It does not delete unknown missing-root workspaces.
## Configuration
@@ -236,21 +191,21 @@ OpenCode Working Memory works out of the box.
Default behavior:
- Workspace memory budget: 3600 characters (~900 tokens)
- Workspace memory budget: 5200 characters
- Workspace memory limit: 28 entries
- Hot session state budget: 700 characters (~175 tokens)
- Hot session state budget: 1200 characters
- Active files shown: 8
- Open errors shown: 3
See [Configuration](docs/configuration.md) for customization options.
## Roadmap
## Ongoing Work
Current focus:
- Add explicit delete tombstones so removed memories do not get re-extracted.
- Enforce explicit `supersedes` chains for safer replacement of obsolete memories.
- Explore tiered hot/warm/cold storage after the retention model has more real-world data.
- Improve memory recording quality so only durable, useful facts are kept.
- Strengthen deduplication and supersession so stale memories do not pile up.
- Add better forgetting behavior for obsolete decisions, preferences, and project facts.
## Documentation
@@ -266,12 +221,13 @@ cd opencode-working-memory
npm install
npm test
npm run typecheck
npm run cleanup:workspaces -- --dry-run
```
## Requirements
- OpenCode plugin API `>=1.2.0 <2.0.0`
- Node.js >= 22.6.0 (for `memory-diag` CLI, which runs TypeScript with `--experimental-strip-types`)
- Node.js >= 18.0.0
## Limitations
-186
View File
@@ -1,191 +1,5 @@
# Release Notes
## 1.5.4 (2026-05-02)
### Memory Diagnostics Surface Cleanup
This cleanup release keeps the newly published `memory-diag` CLI small before legacy spellings become compatibility debt.
### What Changed
- **Official commands only**: the public CLI surface is `status`, `rejected`, `missing`, and `explain`.
- **Pre-public aliases removed**: old spellings such as `health`, `quality`, `rejections`, `disappearances`, and `trace` are no longer recognized.
- **Maintainer diagnostics clarified**: hidden `coverage` and `audit` commands remain available as maintainer-only diagnostics and stay out of public usage output.
- **Cleaner internals**: current command metadata now has one source of truth, and legacy command/formatter wrappers were removed.
- **Internal cleanup script hidden**: the workspace cleanup helper is no longer exposed as an npm script; the underlying development tool remains in `scripts/dev/`.
### Validation
- `npm run typecheck` — `TYPECHECK_PASS`
- `npm test` — 347 tests passing, `TEST_PASS`
---
## 1.5.3 (2026-05-02)
### Published Memory Diagnostics CLI
This release makes the read-only workspace memory diagnostics CLI available as the package binary `memory-diag`, with user-facing commands for checking memory health and understanding why memories are rejected, missing, shown, or hidden.
> Good memory is selective memory — and diagnostics should explain the selection.
### What Changed
- **Published CLI bin**: run diagnostics with `npx --package opencode-working-memory memory-diag` or `memory-diag status`.
- **User-facing commands**: `status`, `rejected`, `missing`, and `explain <memory-id>` are documented as the supported public workflow.
- **Legacy compatibility**: existing maintainer and legacy commands remain accepted with deprecation notices where applicable.
- **Cleaner CLI architecture**: the former monolithic diagnostics script is now split into focused command, model, formatter, and utility modules.
- **Faster diagnostics**: evidence grouping avoids repeated per-memory evidence queries in snapshot diagnostics.
- **Cleaner failures**: top-level CLI error handling now reports usage and unexpected command errors without noisy stack traces.
- **Docs alignment**: README and configuration docs now use package-qualified `npx` commands to avoid resolving an unrelated package named `memory-diag`.
### Requirements
- Node.js `>=22.6.0` is now required because the published diagnostics binary runs TypeScript through Node's `--experimental-strip-types` support.
### Validation
- `npm run typecheck` — `TYPECHECK_PASS`
- `npm test` — 358 tests passing, `TEST_PASS`
---
## 1.5.1 (2026-04-30)
### Evidence Loop and Explainability
This release adds an evidence-based audit trail for memory lifecycle events and user-facing diagnostics for understanding why memories are rendered, promoted, or rejected.
> **Evidence before sublimation.** Every memory decision can be traced.
### What Changed
- **Evidence log**: extraction, promotion, reinforcement, render, and storage events are now recorded in a per-workspace `events.jsonl` with 90-day retention and 5000-event cap.
- **User explainability**: `memory-diag explain` shows per-memory render status with strength, reasons, and evidence. `memory-diag trace --memory <id>` shows the full lifecycle history.
- **Machine-readable diagnostics**: `memory-diag health --json` outputs structured `MemoryDiagJSON` for scripting.
- **Calendar-day reinforcement gate**: reinforcement now requires distinct UTC calendar days, preventing repetitive-task gaming that could inflate a memory's strength within a single day.
- **SafetyCritical deprecation complete**: the `safetyCritical` field no longer affects retention strength or type-cap bypass. All memories fade by the same rules.
- **Retention module extraction**: retention constants and calculations moved to `src/retention.ts` for cleaner separation.
### Privacy
- Evidence text previews are credential-redacted. Memory content is stored as truncated hashes, never in full.
- Diagnostics default to redacted output. `--raw` is available for maintainers.
### Upgrade Notes
- No configuration changes required.
- Existing workspace memory files remain compatible.
- Evidence logs are created automatically; no migration needed.
### Validation
- `npm run typecheck`
- `npm test` — 271 tests passing
---
## 1.5.0 (2026-04-29)
### Retention Decay Model
This release changes workspace memory retention from hard stale pruning and additive priority scoring to a strength-based decay model.
Think of it like a forgetting curve: memories fade over time, but important and reinforced memories decay slower. Weak entries fall out of rendered prompt context by cap competition, not hard deletion.
> **Memory should fade, so the agent can keep learning.**
> Important memories decay slower, but every memory must leave room for newer project reality and avoid long-term memory pollution.
```text
strength
│
██ │╲____ reinforced: slower decline
│ ╲______
▒▒ │ ╲__ ordinary memory
│ ╲
├ ─ ─ ─ ─ ─ ─ ─ ─╲─ dynamic cap competition zone
░░ │ ╲ easier for new memories to replace
│ ↑ still stored, not deleted
└──────────────────────────────→ time / sessions
```
### What Changed
- **Strength-based retention**: workspace memory now uses exponential decay: initial strength × age decay.
- **Better initial strength**: type, source, and user importance now determine how strong a memory starts.
- **No confidence scoring**: confidence remains in stored data for compatibility, but it no longer affects retention ranking.
- **Type caps**: rendered workspace memory now caps feedback, decisions, project facts, and references separately so one type cannot monopolize all 28 slots.
- **Deprecation:** `safetyCritical` field no longer affects retention strength or type-cap bypass. All system memories now fade according to the same rules. Safety rules belong in user-controlled `agent.md` files, not in system memory.
- **Dormant-aware age**: after 14 inactive days, additional dormant workspace time counts at 0.25x so paused projects do not forget too aggressively.
- **Reinforcement**: repeated matching memories reinforce the survivor and slow future decay, with same-session and one-hour guards to avoid accidental spam.
- **No hard stale pruning**: old or stale-marked memories are no longer automatically dropped by age; they lose rendered space only through cap competition.
- **Calibrated prompt budgets**: observed rendered output was typically under ~2000 characters for workspace memory and ~500 characters for hot session state, so defaults were reduced to 3600 and 700 characters to keep overhead lower while retaining buffer.
- **Clearer health output**: `memory-diag health` now reports stored vs rendered counts, type caps, global cap overflow, dormancy, retention monitoring, and strength-ranked top/weakest entries.
### Why This Helps
- User preferences and explicit memories are less likely to disappear just because inferred project facts are newer.
- Feedback, decisions, project facts, and references share prompt space more fairly.
- Returning to an old workspace is less punishing because dormant time decays more slowly.
- Maintainers can see why memories are rendered or capped instead of guessing from a single active-memory count.
- Stale entries can fade out of prompt context without destructive cleanup.
### Diagnostics
Maintainers can inspect retention behavior with:
```bash
bun scripts/memory-diag.ts health
```
The health output now includes sections like:
```txt
Stored active memories: 28
Rendered candidates: 20
By type:
feedback stored=17 rendered=10 typeCap=10
decision stored=11 rendered=10 typeCap=10
Retention caps:
type-capped entries: 8
global-cap overflow: 0
Dormancy:
dormant discount active: no
Retention monitoring:
high_importance_ratio: 0.0% (alert > 30%)
```
### Not Included Yet
- Delete tombstones are not implemented in this release.
- Explicit `supersedes` chain enforcement is still deferred.
- Hot/warm/cold tiered storage remains future work.
### Upgrade Notes
- No configuration changes required.
- Existing workspace memory files remain compatible.
- Existing entries without a `retentionClock` fall back safely to existing timestamps.
- The OpenCode config entry stays the same:
```json
{
"plugin": ["opencode-working-memory"]
}
```
### Validation
- `npm run typecheck`
- `npm test` — 242 tests passing
- `bun scripts/memory-diag.ts health`
---
## 1.4.0 (2026-04-28)
### Memory Quality Cleanup
+11 -49
View File
@@ -10,7 +10,7 @@ OpenCode Working Memory implements a **three-layer memory architecture** designe
│ • Persistent storage: ~/.local/share/opencode-working-... │
│ • Types: feedback | project | decision | reference │
│ • Sources: explicit | compaction | manual │
│ • Render limits: 3600 chars / 28 entries │
│ • Limits: 5200 chars / 28 entries │
│ • Survives: session reset, compaction (same workspace) │
└─────────────────────────────────────────────────────────────┘
↓
@@ -48,9 +48,8 @@ Long-term memory that persists across sessions within the same workspace. Perfec
{
version: 1,
workspace: { root: string, key: string },
limits: { maxRenderedChars: 3600, maxEntries: 28 },
limits: { maxRenderedChars: 5200, maxEntries: 28 },
entries: LongTermMemoryEntry[],
lastActivityAt?: string,
updatedAt: string
}
```
@@ -91,51 +90,18 @@ Memory candidates:
- Path-heavy facts (>50% paths)
- Very short text (<20 chars)
### Consolidation, Deduplication, and Retention
### Consolidation and Deduplication
Memories are deduplicated and consolidated with accounting:
1. Normalize exact text: lowercase, strip punctuation, collapse whitespace.
2. Group project/reference entries by identity where possible.
3. Keep decision and feedback entries on exact canonical matching to avoid broad semantic merges.
4. Keep the best surviving entry by source, confidence, specificity, and freshness tie-breakers.
3. Group decisions and feedback by topic where possible.
4. Keep the best surviving entry by source, confidence, type, and freshness rules.
5. Emit accounting events so pending memories can be classified as promoted, absorbed, superseded, or rejected.
This prevents absorbed or superseded pending memories from retrying forever while still preserving the active surviving memory.
Retention then decides which active memories are rendered into prompt context. It does not hard-delete old memories by age.
```typescript
strength = initialStrength * 2 ** (-effectiveAgeDays / effectiveHalfLifeDays)
```
Initial strength is based on memory type, source, and optional user importance. Confidence remains stored for compatibility but is not part of retention scoring.
Rendered candidates are selected in this order:
1. Exclude `status: "superseded"` entries.
2. Compute current retention strength.
3. Sort by strength descending.
4. Apply per-type caps.
5. Keep the top 28 rendered entries under the workspace memory character budget.
Default type caps:
| Type | Rendered cap |
|------|--------------|
| `feedback` | 10 |
| `decision` | 10 |
| `project` | 8 |
| `reference` | 6 |
The type-cap total is 34, intentionally above the global 28-entry cap. These are maximums, not quotas.
Dormant workspaces age more slowly: after 14 inactive days, additional dormant time counts at 0.25x for retention decay. Repeated duplicate memories reinforce the surviving entry and slow future decay, but same-session and under-one-hour repeats do not stack reinforcement.
### Safety-Critical Deprecation
The `safetyCritical` field on `LongTermMemoryEntry` is deprecated as of the retention v1.5.1 model update. It no longer affects retention strength or type-cap bypass. The field is preserved in the type definition for backward compatibility with existing workspace memory JSON files, but has no active behavior. Safety rules should be maintained in user-controlled files such as `agent.md` rather than in system memory.
### System Prompt Injection
Workspace memory is injected at the top of every message:
@@ -275,7 +241,7 @@ Applies quality gate, redaction, migration, consolidation accounting, deduplicat
- `session.compacted`: Promote session decisions to workspace memory
- `session.deleted`: Clean up session state files
Promotion uses accounting results from workspace memory normalization. Pending memories that are kept are promoted; duplicate memories are absorbed; exact decision replacements can be superseded; over-capacity compaction memories are rejected. Stale-marked memories are not hard-pruned by age; they lose rendered space through retention strength and cap competition.
Promotion uses accounting results from workspace memory normalization. Pending memories that are kept are promoted; duplicate memories are absorbed; obsolete same-topic memories are superseded; stale or over-capacity compaction memories are rejected.
## Quality Guarantees
@@ -347,24 +313,20 @@ canonical("Use npm cache for plugins") === "use npm cache for plugins"
const workspaceKey = sha256(realpath(workspaceRoot)).slice(0, 16)
```
### Storage Safety
All read-modify-write updates go through `updateJSON()`, which combines an in-process promise queue with an on-disk `.lock` file. The lock file uses exclusive creation, heartbeat refreshes while held, stale-lock recovery after 30 seconds, and a 5 second wait timeout for live contention. Direct `readJSON()` is fallback-oriented and does not mutate data except when corrupt JSON is quarantined.
## Performance Considerations
### Memory Budgets
| Layer | Max Chars | Max Entries |
|-------|-----------|-------------|
| Workspace Memory | 3600 | 28 |
| Hot Session State | 700 | 8 files, 3 errors |
| Workspace Memory | 5200 | 28 |
| Hot Session State | 1200 | 8 files, 3 errors |
### Injection Overhead
- Workspace memory: usually under ~2000 chars in observed rendered output
- Hot session state: usually under ~500 chars in observed rendered output
- Total: typically well below the configured maximums
- Workspace memory: ~200-500 chars per message
- Hot session state: ~200-400 chars per message
- Total: ~400-900 chars per message (minimal)
### Storage Footprint
+11 -56
View File
@@ -8,8 +8,8 @@ OpenCode Working Memory works out-of-the-box with sensible defaults. Configurati
```typescript
const LONG_TERM_LIMITS = {
maxRenderedChars: 3600, // Maximum characters in system prompt
targetRenderedChars: 3000, // Target characters (leave buffer)
maxRenderedChars: 5200, // Maximum characters in system prompt
targetRenderedChars: 4200, // Target characters (leave buffer)
maxEntries: 28, // Maximum number of entries
maxEntryTextChars: 260, // Maximum characters per entry text
maxRationaleChars: 180, // Maximum characters per entry rationale
@@ -18,40 +18,14 @@ const LONG_TERM_LIMITS = {
**Recommendations**:
- Keep `maxRenderedChars` under 5500 to avoid context bloat
- Defaults are calibrated from observed rendered usage that was typically under ~2000 characters
- `maxEntries` of 28 provides good coverage without overwhelming
- Entry text limits ensure entries stay concise
## Retention Model Defaults
Workspace memory retention uses strength-based decay. These constants live in `src/workspace-memory.ts`:
```typescript
const BASE_HALF_LIFE_DAYS = 45;
const REINFORCEMENT_HALFLIFE_FACTOR = 0.85;
const REINFORCEMENT_MAX_COUNT = 6;
const WORKSPACE_DORMANT_AFTER_DAYS = 14;
const DORMANT_DECAY_MULTIPLIER = 0.25;
```
Initial strength uses type, source, and user importance factors. Confidence is stored for compatibility but is not used for retention scoring.
Rendered type caps prevent one type from filling all workspace memory slots:
| Type | Rendered cap |
|------|--------------|
| `feedback` | 10 |
| `decision` | 10 |
| `project` | 8 |
| `reference` | 6 |
Old or stale-marked memories are not hard-pruned by age; they lose rendered space through strength and cap competition. The deprecated `safetyCritical` field is preserved for compatibility but no longer affects strength or type caps.
## Hot Session State Limits
```typescript
const HOT_STATE_LIMITS = {
maxRenderedChars: 700, // Maximum characters in system prompt
maxRenderedChars: 1200, // Maximum characters in system prompt
maxActiveFilesStored: 20, // Maximum files tracked in state
maxActiveFilesRendered: 8, // Maximum files shown in prompt
maxOpenErrorsStored: 5, // Maximum errors tracked
@@ -62,7 +36,6 @@ const HOT_STATE_LIMITS = {
**Recommendations**:
- Keep `maxRenderedChars` under 1500 for fast prompts
- Defaults are calibrated from observed rendered usage around ~500 characters or less
- `maxActiveFilesRendered` of 8 provides good context coverage
- `maxOpenErrorsRendered` of 3 avoids overwhelming error lists
@@ -70,12 +43,12 @@ const HOT_STATE_LIMITS = {
### Long-Term Memory Types
| Type | Purpose | Rendered cap |
|------|---------|--------------|
| `feedback` | User preferences for workspace | 10 |
| `project` | Project-level information | 8 |
| `decision` | Important decisions | 10 |
| `reference` | Key references | 6 |
| Type | Purpose | Stale After (days) |
|------|---------|---------------------|
| `feedback` | User preferences for workspace | 90 |
| `project` | Project-level information | 60 |
| `decision` | Important decisions | 45 |
| `reference` | Key references | 90 |
### Memory Sources
@@ -141,7 +114,7 @@ To customize limits, edit the constants in `src/types.ts`:
```typescript
// Example: Increase workspace memory limit
export const LONG_TERM_LIMITS = {
maxRenderedChars: 6000, // Increased from 3600
maxRenderedChars: 6000, // Increased from 5200
maxEntries: 35, // Increased from 28
// ...
};
@@ -171,7 +144,7 @@ const HOT_STATE_LIMITS = {
// Preserve more context
const LONG_TERM_LIMITS = {
maxEntries: 40, // Increased
targetRenderedChars: 5000, // Increased
targetRenderedChars: 5000, // Increased
};
```
@@ -202,24 +175,6 @@ cat ~/.local/share/opencode-working-memory/workspaces/*/workspace-memory.json |
cat ~/.local/share/opencode-working-memory/workspaces/*/sessions/*.json | jq
```
### Inspect Retention Health
Use the diagnostics CLI to check memory health for the current workspace:
```bash
npx --package opencode-working-memory memory-diag status
# or from a source checkout:
npm run diag -- status
```
For detailed diagnostics, use `--verbose`:
```bash
npx --package opencode-working-memory memory-diag status --verbose
```
The status output includes overall health (OK/WARNING/DEGRADED), key metrics, and suggested next steps when attention is needed.
### Clear Workspace Memory
```bash
+5 -11
View File
@@ -1,29 +1,23 @@
{
"name": "opencode-working-memory",
"version": "1.5.4",
"version": "1.4.0",
"description": "Three-layer memory architecture for OpenCode with workspace memory and hot session state",
"type": "module",
"main": "index.ts",
"exports": {
".": "./index.ts"
},
"bin": {
"memory-diag": "./scripts/memory-diag-bin.cjs"
},
"files": [
"index.ts",
"src/",
"scripts/memory-diag.ts",
"scripts/memory-diag/",
"scripts/memory-diag-bin.cjs",
"README.md",
"LICENSE"
],
"scripts": {
"build": "node -e \"console.log('No build step required: OpenCode loads index.ts directly')\"",
"diag": "node --experimental-strip-types scripts/memory-diag.ts",
"typecheck": "tsc --noEmit && node -e \"console.log('TYPECHECK_PASS')\"",
"test": "node --import ./tests/setup-xdg-data-home.ts --test --experimental-strip-types tests/*.test.ts && node -e \"console.log('TEST_PASS')\"",
"typecheck": "tsc --noEmit",
"test": "node --import ./tests/setup-xdg-data-home.ts --test --experimental-strip-types tests/*.test.ts",
"cleanup:workspaces": "node --experimental-strip-types scripts/dev/cleanup-workspaces.ts",
"check:compat": "npm install --no-save @opencode-ai/plugin@latest && npm run typecheck && npm test"
},
"keywords": [
@@ -53,6 +47,6 @@
"typescript": "^5.9.2"
},
"engines": {
"node": ">=22.6.0"
"node": ">=18.0.0"
}
}
-368
View File
@@ -1,368 +0,0 @@
#!/usr/bin/env node
/**
* Read-only baseline report for workspace-memory exact/identity key collisions.
*
* This script reads workspace-memory.json files from the local workspace memory
* data directory. It never writes to workspace memory stores; the only optional
* write is the explicit --json report path.
*/
import { existsSync } from "node:fs";
import { mkdir, readdir, readFile, writeFile } from "node:fs/promises";
import { homedir } from "node:os";
import { basename, dirname, join, resolve } from "node:path";
import { workspaceMemoryExactKey, workspaceMemoryIdentityKey } from "../../src/workspace-memory.ts";
import type { LongTermMemoryEntry, LongTermSource, LongTermType } from "../../src/types.ts";
type CliOptions = {
workspacesDir: string;
jsonPath?: string;
};
type ReinforcementCountStatistics = {
min: number;
max: number;
mean: number;
gtZero: number;
};
type MemorySummary = {
id: string;
type: LongTermType;
source: LongTermSource;
text: string;
reinforcementCount: number;
};
type WorkspaceSummary = {
dirName: string;
dirPath: string;
key: string;
root: string;
};
type CollisionGroup = {
kind: "exact" | "identity";
workspace: WorkspaceSummary;
key: string;
classification: "unclassified";
memories: MemorySummary[];
};
type ValidationReport = {
summary: {
workspaces: number;
activeMemories: number;
exactCollisionGroups: number;
identityCollisionGroups: number;
reinforcementCountStatistics: ReinforcementCountStatistics;
};
groups: CollisionGroup[];
};
type ScannedMemory = {
entry: LongTermMemoryEntry;
exactKey: string;
identityKey: string;
reinforcementCount: number;
};
const DEFAULT_WORKSPACES_DIR = join(homedir(), ".local", "share", "opencode-working-memory", "workspaces");
const WORKSPACES_DIR_ENV_KEYS = [
"OPENCODE_WORKING_MEMORY_WORKSPACES_DIR",
"WORKSPACE_MEMORY_WORKSPACES_DIR",
];
function usage(): string {
return `Usage:
node --experimental-strip-types scripts/dev/validate-identity-keys.ts [workspaces-dir] [--json <path>]
node --experimental-strip-types scripts/dev/validate-identity-keys.ts --data-path <workspaces-dir> [--json <path>]
Defaults:
workspaces-dir defaults to ${DEFAULT_WORKSPACES_DIR}
env override: ${WORKSPACES_DIR_ENV_KEYS.join(" or ")}
`;
}
function die(message: string): never {
console.error(message);
console.error(usage());
process.exit(1);
}
function envWorkspacesDir(): string | undefined {
for (const key of WORKSPACES_DIR_ENV_KEYS) {
const value = process.env[key];
if (value) return value;
}
return undefined;
}
function expandHome(path: string): string {
if (path === "~") return homedir();
if (path.startsWith("~/")) return join(homedir(), path.slice(2));
return path;
}
function parseArgs(argv: string[]): CliOptions {
let workspacesDir: string | undefined;
let jsonPath: string | undefined;
for (let i = 0; i < argv.length; i += 1) {
const arg = argv[i];
if (arg === "--help" || arg === "-h") {
console.log(usage());
process.exit(0);
}
if (arg === "--json") {
const value = argv[++i];
if (!value) die("--json requires a path");
jsonPath = expandHome(value);
continue;
}
if (arg === "--data-path" || arg === "--workspaces-dir") {
const value = argv[++i];
if (!value) die(`${arg} requires a path`);
if (workspacesDir) die("Only one workspaces directory may be provided");
workspacesDir = expandHome(value);
continue;
}
if (arg.startsWith("--")) die(`Unknown option: ${arg}`);
if (workspacesDir) die("Only one workspaces directory may be provided");
workspacesDir = expandHome(arg);
}
return {
workspacesDir: resolve(workspacesDir ?? expandHome(envWorkspacesDir() ?? DEFAULT_WORKSPACES_DIR)),
jsonPath: jsonPath ? resolve(jsonPath) : undefined,
};
}
function isLongTermType(value: unknown): value is LongTermType {
return value === "feedback" || value === "project" || value === "decision" || value === "reference";
}
function isLongTermSource(value: unknown): value is LongTermSource {
return value === "explicit" || value === "compaction" || value === "manual";
}
function isMemoryEntry(value: unknown): value is LongTermMemoryEntry {
if (!value || typeof value !== "object") return false;
const entry = value as Partial<LongTermMemoryEntry>;
return typeof entry.id === "string"
&& isLongTermType(entry.type)
&& typeof entry.text === "string"
&& isLongTermSource(entry.source);
}
function reinforcementCount(entry: LongTermMemoryEntry): number {
const count = entry.reinforcementCount ?? 0;
return Number.isFinite(count) && count > 0 ? count : 0;
}
function summarizeMemory(memory: ScannedMemory): MemorySummary {
return {
id: memory.entry.id,
type: memory.entry.type,
source: memory.entry.source,
text: memory.entry.text,
reinforcementCount: memory.reinforcementCount,
};
}
function addToGroupMap(map: Map<string, ScannedMemory[]>, key: string, memory: ScannedMemory): void {
const group = map.get(key);
if (group) {
group.push(memory);
} else {
map.set(key, [memory]);
}
}
function computeReinforcementStatistics(counts: number[]): ReinforcementCountStatistics {
if (counts.length === 0) {
return { min: 0, max: 0, mean: 0, gtZero: 0 };
}
const sum = counts.reduce((total, count) => total + count, 0);
return {
min: Math.min(...counts),
max: Math.max(...counts),
mean: sum / counts.length,
gtZero: counts.filter(count => count > 0).length,
};
}
async function readWorkspaceMemory(workspaceDir: string): Promise<unknown | undefined> {
const path = join(workspaceDir, "workspace-memory.json");
if (!existsSync(path)) return undefined;
try {
const raw = await readFile(path, "utf8");
return JSON.parse(raw);
} catch (error) {
const code = error && typeof error === "object" && "code" in error ? String(error.code) : "";
if (code === "ENOENT" || code === "EISDIR") return undefined;
throw error;
}
}
async function scanWorkspaces(workspacesDir: string): Promise<ValidationReport> {
const dirents = await readdir(workspacesDir, { withFileTypes: true });
const workspaceDirs = dirents
.filter(dirent => dirent.isDirectory())
.map(dirent => dirent.name)
.sort((a, b) => a.localeCompare(b));
const collisionGroups: CollisionGroup[] = [];
const reinforcementCounts: number[] = [];
let workspaces = 0;
let activeMemories = 0;
for (const dirName of workspaceDirs) {
const dirPath = join(workspacesDir, dirName);
const store = await readWorkspaceMemory(dirPath);
if (!store || typeof store !== "object") continue;
const storeObject = store as {
workspace?: { key?: unknown; root?: unknown };
entries?: unknown;
};
const rawEntries = Array.isArray(storeObject.entries) ? storeObject.entries : [];
const workspace: WorkspaceSummary = {
dirName,
dirPath,
key: typeof storeObject.workspace?.key === "string" ? storeObject.workspace.key : dirName,
root: typeof storeObject.workspace?.root === "string" ? storeObject.workspace.root : "<unknown>",
};
workspaces += 1;
const exactGroups = new Map<string, ScannedMemory[]>();
const identityGroups = new Map<string, ScannedMemory[]>();
for (const rawEntry of rawEntries) {
if (!isMemoryEntry(rawEntry)) continue;
if (rawEntry.status === "superseded") continue;
const scanned: ScannedMemory = {
entry: rawEntry,
exactKey: workspaceMemoryExactKey(rawEntry),
identityKey: workspaceMemoryIdentityKey(rawEntry),
reinforcementCount: reinforcementCount(rawEntry),
};
activeMemories += 1;
reinforcementCounts.push(scanned.reinforcementCount);
addToGroupMap(exactGroups, scanned.exactKey, scanned);
addToGroupMap(identityGroups, scanned.identityKey, scanned);
}
for (const [key, memories] of exactGroups.entries()) {
if (memories.length < 2) continue;
collisionGroups.push({
kind: "exact",
workspace,
key,
classification: "unclassified",
memories: memories.map(summarizeMemory),
});
}
for (const [key, memories] of identityGroups.entries()) {
if (memories.length < 2) continue;
collisionGroups.push({
kind: "identity",
workspace,
key,
classification: "unclassified",
memories: memories.map(summarizeMemory),
});
}
}
const exactCollisionGroups = collisionGroups.filter(group => group.kind === "exact").length;
const identityCollisionGroups = collisionGroups.filter(group => group.kind === "identity").length;
return {
summary: {
workspaces,
activeMemories,
exactCollisionGroups,
identityCollisionGroups,
reinforcementCountStatistics: computeReinforcementStatistics(reinforcementCounts),
},
groups: collisionGroups,
};
}
function oneLine(text: string): string {
return text.replace(/\s+/g, " ").trim();
}
function renderHumanReport(report: ValidationReport, workspacesDir: string): string {
const stats = report.summary.reinforcementCountStatistics;
const lines: string[] = [];
lines.push("Workspace memory identity key validation baseline");
lines.push(`Data path: ${workspacesDir}`);
lines.push("");
lines.push("Summary:");
lines.push(`- Workspaces scanned: ${report.summary.workspaces}`);
lines.push(`- Active memories: ${report.summary.activeMemories}`);
lines.push(`- Exact collision groups: ${report.summary.exactCollisionGroups}`);
lines.push(`- Identity collision groups: ${report.summary.identityCollisionGroups}`);
lines.push(`- Reinforcement counts: min=${stats.min} max=${stats.max} mean=${stats.mean.toFixed(2)} gtZero=${stats.gtZero}`);
if (report.groups.length === 0) {
lines.push("");
lines.push("Collision breakdown: none");
return lines.join("\n");
}
const groupsByWorkspace = new Map<string, CollisionGroup[]>();
for (const group of report.groups) {
const key = group.workspace.key;
const existing = groupsByWorkspace.get(key);
if (existing) existing.push(group);
else groupsByWorkspace.set(key, [group]);
}
lines.push("");
lines.push("Collision breakdown:");
for (const [workspaceKey, groups] of [...groupsByWorkspace.entries()].sort((a, b) => a[0].localeCompare(b[0]))) {
const workspace = groups[0].workspace;
const exactCount = groups.filter(group => group.kind === "exact").length;
const identityCount = groups.filter(group => group.kind === "identity").length;
lines.push(`- Workspace ${workspaceKey}`);
lines.push(` root: ${workspace.root}`);
lines.push(` dir: ${workspace.dirPath}`);
lines.push(` exactCollisionGroups=${exactCount} identityCollisionGroups=${identityCount}`);
for (const group of groups.sort((a, b) => a.kind.localeCompare(b.kind) || a.key.localeCompare(b.key))) {
lines.push(` - ${group.kind} key: ${group.key}`);
lines.push(` classification: ${group.classification}`);
for (const memory of group.memories) {
lines.push(` - id=${memory.id} type=${memory.type} source=${memory.source} reinforcementCount=${memory.reinforcementCount}`);
lines.push(` text: ${oneLine(memory.text)}`);
}
}
}
return lines.join("\n");
}
async function writeJsonReport(path: string, report: ValidationReport): Promise<void> {
if (basename(path) === "workspace-memory.json") {
die("Refusing to write JSON output to a workspace-memory.json file");
}
await mkdir(dirname(path), { recursive: true });
await writeFile(path, `${JSON.stringify(report, null, 2)}\n`, "utf8");
}
const options = parseArgs(process.argv.slice(2));
const report = await scanWorkspaces(options.workspacesDir);
console.log(renderHumanReport(report, options.workspacesDir));
if (options.jsonPath) {
await writeJsonReport(options.jsonPath, report);
console.log(`\nJSON report written to: ${options.jsonPath}`);
}
-24
View File
@@ -1,24 +0,0 @@
#!/usr/bin/env node
const { execFileSync } = require("child_process");
const path = require("path");
function isSupportedNodeVersion(version) {
const [major = 0, minor = 0, patch = 0] = version.split(".").map(part => Number(part));
void patch;
return major > 22 || (major === 22 && minor >= 6);
}
if (!isSupportedNodeVersion(process.versions.node)) {
process.stderr.write(`memory-diag requires Node >=22.6.0 because it runs TypeScript with --experimental-strip-types. Current Node: v${process.versions.node}.\n`);
process.exit(1);
}
const binDir = __dirname;
const tsScript = path.join(binDir, "memory-diag.ts");
const args = ["--experimental-strip-types", tsScript, ...process.argv.slice(2)];
try {
execFileSync(process.execPath, args, { stdio: "inherit" });
process.exit(0);
} catch (e) {
process.exit(e.status || 1);
}
-54
View File
@@ -1,54 +0,0 @@
#!/usr/bin/env bun
/**
* Maintainer-only offline diagnostics for memory quality calibration.
* Does not send telemetry, make API calls, or affect plugin runtime behavior.
*/
import { parseArgs, type ParsedArgs } from "./memory-diag/cli.ts";
import { dispatch } from "./memory-diag/command-registry.ts";
import { CliInputError } from "./memory-diag/types.ts";
type ParsedError = Extract<ParsedArgs, { ok: false }>;
type ParsedHelp = Extract<ParsedArgs, { ok: true; help: true }>;
function isParsedError(parsed: ParsedArgs): parsed is ParsedError {
return parsed.ok === false;
}
function isParsedHelp(parsed: ParsedArgs): parsed is ParsedHelp {
return parsed.ok === true && "help" in parsed;
}
async function main(): Promise<number> {
const parsed = parseArgs(process.argv.slice(2));
if (isParsedError(parsed)) {
console.error(parsed.message);
console.error(parsed.usage);
return parsed.exitCode;
}
if (isParsedHelp(parsed)) {
console.log(parsed.usage);
return 0;
}
try {
const result = await dispatch(parsed.command, parsed.options);
const stderr = [parsed.deprecationNotice, result.stderr].filter(Boolean).join("\n");
if (stderr) process.stderr.write(stderr.endsWith("\n") ? stderr : `${stderr}\n`);
if (result.stdout) process.stdout.write(result.stdout.endsWith("\n") ? result.stdout : `${result.stdout}\n`);
return result.exitCode ?? 0;
} catch (error) {
if (error instanceof CliInputError) {
process.stderr.write(`${error.message}\n`);
return 1;
}
if (error instanceof Error) {
process.stderr.write(`memory-diag failed: ${error.message}\n`);
return 1;
}
process.stderr.write(`memory-diag failed: ${String(error)}\n`);
return 1;
}
}
process.exitCode = await main();
-124
View File
@@ -1,124 +0,0 @@
import { HIDDEN_COMMAND_NOTICES, isCommand } from "./command-metadata.ts";
import type { CliOptions, Command, ParsedArgs } from "./types.ts";
export type { ParsedArgs } from "./types.ts";
export function usage(): string {
return `Usage:
memory-diag [status] [--workspace <path>] [--verbose] [--json]
memory-diag rejected [--workspace <path>] [--verbose] [--json]
memory-diag missing [--workspace <path>] [--verbose] [--json]
memory-diag explain [memory-id] [--workspace <path>] [--raw]
Global options:
--workspace <path> Workspace path (default: current directory)
--verbose Show detailed diagnostics
--json Print machine-readable JSON where supported
--no-emoji Disable emoji in human output
`;
}
function error(message: string): ParsedArgs {
return { ok: false, message, usage: usage(), exitCode: 1 };
}
function isValidSince(rawSince: string): boolean {
if (/^(\d+)([dhm])$/i.test(rawSince)) return true;
return !Number.isNaN(new Date(rawSince).getTime());
}
export function parseArgs(argv: string[]): ParsedArgs {
const [first, ...tail] = argv;
if (first === "--help" || first === "-h") {
return { ok: true, help: true, usage: usage() };
}
let command: Command = "status";
let deprecationNotice: string | undefined;
let rest = argv;
if (first && !first.startsWith("--")) {
rest = tail;
if (isCommand(first)) {
command = first;
deprecationNotice = HIDDEN_COMMAND_NOTICES[first];
} else {
return error(`Unknown subcommand: ${first}`);
}
}
const options: CliOptions = { raw: false, positional: [] };
for (let i = 0; i < rest.length; i += 1) {
const arg = rest[i];
if (arg === "--raw") options.raw = true;
else if (arg === "--json") options.json = true;
else if (arg === "--verbose") options.verbose = true;
else if (arg === "--no-emoji") options.noEmoji = true;
else if (arg === "--all") options.all = true;
else if (arg === "--soft-only") options.softOnly = true;
else if (arg === "--trigger-only") options.triggerOnly = true;
else if (arg === "--include-historical") options.includeHistorical = true;
else if (arg === "--explain") options.explain = true;
else if (arg === "--workspace") {
const value = rest[++i];
if (!value) return error("--workspace requires a path");
options.workspace = value;
} else if (arg === "--since") {
const value = rest[++i];
if (!value) return error("--since requires a duration or ISO timestamp");
options.since = value;
} else if (arg === "--reason") {
const value = rest[++i];
if (!value) return error("--reason requires a reason code");
options.reason = value;
} else if (arg === "--migration") {
const value = rest[++i];
if (!value) return error("--migration requires an id");
options.migration = value;
} else if (arg === "--memory") {
const value = rest[++i];
if (!value) return error("--memory requires an id");
options.memory = value;
} else if (!arg.startsWith("--") && command === "explain") {
options.positional?.push(arg);
} else {
return error(`Unknown option: ${arg}`);
}
}
if (command === "explain") {
const positional = options.positional ?? [];
if (positional.length > 1) return error("explain accepts at most one memory id");
if (positional.length === 1 && options.memory) return error("Use either explain <memory-id> or --memory, not both");
if (positional.length === 1) options.memory = positional[0];
} else if ((options.positional ?? []).length > 0) {
return error(`Unknown option: ${options.positional?.[0]}`);
}
if (command === "status") {
if (options.all) return error(`${command} does not accept --all`);
} else if (command === "rejected" || command === "missing" || command === "coverage" || command === "explain") {
if (options.all) return error(`${command} does not accept --all`);
} else {
if (options.all || options.workspace) return error(`${command} does not accept --all or --workspace`);
}
if (options.json && command !== "status" && command !== "rejected" && command !== "missing" && command !== "coverage") {
return error(`${command} does not accept --json`);
}
if (command !== "rejected" && (options.softOnly || options.triggerOnly || options.since)) {
return error(`${command} does not accept rejection filters`);
}
if (command !== "coverage" && options.includeHistorical) return error(`${command} does not accept --include-historical`);
if (command !== "rejected" && options.reason) return error(`${command} does not accept rejection filters`);
if (command !== "missing" && options.explain) return error(`${command} does not accept --explain`);
if (command !== "audit" && options.migration) {
return error(`${command} does not accept --migration`);
}
if (command !== "explain" && options.memory) {
return error(`${command} does not accept --memory`);
}
if (command === "rejected" && options.since && !isValidSince(options.since)) {
return error(`Invalid --since value: ${options.since}`);
}
return { ok: true, command, options, deprecationNotice };
}
-16
View File
@@ -1,16 +0,0 @@
export const VISIBLE_COMMANDS = ["status", "rejected", "missing", "explain"] as const;
export const HIDDEN_COMMANDS = ["coverage", "audit"] as const;
export type VisibleCommand = typeof VISIBLE_COMMANDS[number];
export type HiddenCommand = typeof HIDDEN_COMMANDS[number];
export type Command = VisibleCommand | HiddenCommand;
export const HIDDEN_COMMAND_NOTICES: Partial<Record<Command, string>> = {
coverage: "Note: 'coverage' is a maintainer-only diagnostic and is not part of the public CLI surface.",
audit: "Note: 'audit' is a maintainer-only diagnostic and is not part of the public CLI surface.",
};
export function isCommand(value: string | undefined): value is Command {
return value !== undefined
&& ([...VISIBLE_COMMANDS, ...HIDDEN_COMMANDS] as readonly string[]).includes(value);
}
-18
View File
@@ -1,18 +0,0 @@
import { runAudit } from "./commands/audit.ts";
import { runCoverage } from "./commands/coverage.ts";
import { runExplain } from "./commands/explain.ts";
import { runMissing } from "./commands/missing.ts";
import { runRejected } from "./commands/rejected.ts";
import { runStatus } from "./commands/status.ts";
import type { CliOptions, Command, CommandResult } from "./types.ts";
export async function dispatch(command: Command, options: CliOptions): Promise<CommandResult> {
switch (command) {
case "status": return runStatus(options);
case "rejected": return runRejected(options);
case "missing": return runMissing(options);
case "coverage": return runCoverage(options);
case "audit": return runAudit(options);
case "explain": return runExplain(options);
}
}
-24
View File
@@ -1,24 +0,0 @@
import { HARD_QUALITY_REASONS } from "../../../src/memory-quality.ts";
import { formatMigrationAudit, type MigrationAuditReport } from "../formatters/audit.ts";
import { readJSONLFile } from "../io.ts";
import { migrationIdFromPath, migrationLogPaths, riskySupersedeReasons } from "../migrations-model.ts";
import type { CliOptions, CommandResult, MigrationLogRecord } from "../types.ts";
export async function runAudit(options: CliOptions): Promise<CommandResult> {
const paths = await migrationLogPaths(options);
const reports: MigrationAuditReport[] = [];
for (const path of paths) {
const migrationId = options.migration ?? migrationIdFromPath(path);
const { records, invalidLines } = await readJSONLFile<MigrationLogRecord>(path);
const superseded = records.filter(record => !record.afterStatus || record.afterStatus === "superseded");
const hardReasons = superseded.flatMap(record => {
if (Array.isArray(record.hardReasons)) return record.hardReasons;
return Array.isArray(record.reasons) ? record.reasons.filter(reason => HARD_QUALITY_REASONS.has(reason)) : [];
});
const risky = superseded
.map(record => ({ record, reasons: riskySupersedeReasons(record) }))
.filter(item => item.reasons.length > 0);
reports.push({ migrationId, path, invalidLines, superseded, hardReasons, risky });
}
return { stdout: formatMigrationAudit(reports, { raw: options.raw }) };
}
-14
View File
@@ -1,14 +0,0 @@
import { buildInspectionReadModel, coverageRows } from "../inspection-model.ts";
import { buildCoverageJSON, formatCoverage } from "../formatters/coverage.ts";
import type { CliOptions, CommandResult } from "../types.ts";
export async function runCoverage(options: CliOptions): Promise<CommandResult> {
const model = await buildInspectionReadModel(options);
const rows = coverageRows(model, options.includeHistorical === true);
if (options.json) {
return { stdout: JSON.stringify(buildCoverageJSON(rows), null, 2) };
}
return { stdout: formatCoverage(rows) };
}
-19
View File
@@ -1,19 +0,0 @@
import { traceMemoryLifecycle } from "../../../src/evidence-log.ts";
import { formatExplain } from "../formatters/explain.ts";
import { formatTrace } from "../formatters/trace.ts";
import { snapshotForOptions } from "../workspace-snapshot.ts";
import type { CliOptions, CommandResult } from "../types.ts";
export async function runExplain(options: CliOptions): Promise<CommandResult> {
if (options.memory) {
const root = options.workspace ?? process.cwd();
const [snapshot, trace] = await Promise.all([
snapshotForOptions(options),
traceMemoryLifecycle(root, { memoryId: options.memory }),
]);
return { stdout: formatTrace(options.memory, snapshot, trace) };
}
const snapshot = await snapshotForOptions(options);
return { stdout: formatExplain(snapshot) };
}
-14
View File
@@ -1,14 +0,0 @@
import { buildInspectionReadModel, disappearanceRows } from "../inspection-model.ts";
import { buildMissingJSON, formatMissing } from "../formatters/missing.ts";
import type { CliOptions, CommandResult } from "../types.ts";
export async function runMissing(options: CliOptions): Promise<CommandResult> {
const model = await buildInspectionReadModel(options);
const rows = disappearanceRows(model);
if (options.json) {
return { stdout: JSON.stringify(buildMissingJSON(rows, { explain: options.explain }), null, 2) };
}
return { stdout: formatMissing(rows, { verbose: options.verbose, explain: options.explain }) };
}
-25
View File
@@ -1,25 +0,0 @@
import { buildRejectedJSON, formatRejected } from "../formatters/rejected.ts";
import { loadRejectionRecords, rejectionFalsePositiveRisk, rejectionQualitySummary } from "../rejections-model.ts";
import type { CliOptions, CommandResult } from "../types.ts";
export async function runRejected(options: CliOptions): Promise<CommandResult> {
const { path, invalidLines, records } = await loadRejectionRecords(options);
const summary = rejectionQualitySummary(records);
const falsePositiveRisk = rejectionFalsePositiveRisk(summary);
if (options.json) {
return { stdout: JSON.stringify(buildRejectedJSON({ summary, falsePositiveRisk }), null, 2) };
}
return {
stdout: formatRejected({
path,
invalidLines,
records,
summary,
falsePositiveRisk,
raw: options.raw,
verbose: options.verbose,
}),
};
}
-112
View File
@@ -1,112 +0,0 @@
import { RETENTION_TYPE_MAX, WORKSPACE_DORMANT_AFTER_DAYS } from "../../../src/retention.ts";
import { TYPES } from "../constants.ts";
import { formatStatus, type StatusReadout } from "../formatters/status.ts";
import { buildInspectionReadModel, disappearanceRows } from "../inspection-model.ts";
import { retentionCandidatesForDiag, retentionClockSummary, daysSinceIso } from "../retention-model.ts";
import { rejectionFalsePositiveRisk, rejectionQualitySummary } from "../rejections-model.ts";
import { isWithinDays, memoryDiagJSONFromSnapshot } from "../workspace-snapshot.ts";
import type { CliOptions, CommandResult, MemoryInspectionReadModel } from "../types.ts";
export function buildStatusReadout(model: MemoryInspectionReadModel, now = Date.now()): StatusReadout {
const active = model.store.entries.filter(entry => entry.status !== "superseded");
const retention = retentionCandidatesForDiag(model.store, now);
const clocks = retentionClockSummary(active);
const disappearances = disappearanceRows(model);
const evidenceCovered = active.filter(entry => (model.evidenceByMemoryId.get(entry.id) ?? []).length > 0).length;
const rejectionSummary = rejectionQualitySummary(model.rejectionRecords);
const falsePositiveRisk = rejectionFalsePositiveRisk(rejectionSummary);
const typeCounts = Object.fromEntries(TYPES.map(type => [type, active.filter(entry => entry.type === type).length]));
const capsFull = active.length >= model.store.limits.maxEntries || TYPES.some(type => (typeCounts[type] ?? 0) >= RETENTION_TYPE_MAX[type]);
const unknownDisappearances = disappearances.filter(row => row.classification === "historical_absent_unknown_reason").length;
const status = unknownDisappearances > 0 || clocks.invalid > 0
? "degraded"
: capsFull || rejectionSummary.legacyUnscopedCount > 0 || falsePositiveRisk === "high"
? "warning"
: "ok";
const rejectedLast7Days = model.evidenceEvents.filter(event => event.outcome === "rejected" && isWithinDays(event.createdAt, 7, now)).length;
const evidenceCoveragePercent = active.length === 0 ? 100 : Math.round((evidenceCovered / active.length) * 100);
const needsAttention: string[] = [];
if (clocks.invalid > 0) needsAttention.push(`${clocks.invalid} invalid retention clocks`);
if (unknownDisappearances > 0) needsAttention.push(`${unknownDisappearances} memories disappeared without terminal evidence`);
if (capsFull) needsAttention.push("memory store is at or over retention caps");
if (rejectionSummary.legacyUnscopedCount > 0) needsAttention.push(`${rejectionSummary.legacyUnscopedCount} legacy unscoped rejection records`);
if (falsePositiveRisk === "high") needsAttention.push("possible rejection false-positive risk is high");
const suggestedNextSteps: string[] = [];
if (clocks.invalid > 0 || unknownDisappearances > 0) suggestedNextSteps.push("Run memory-diag missing --verbose to inspect missing-memory evidence.");
if (capsFull) suggestedNextSteps.push("Run memory-diag explain to see which memories are hidden by caps.");
if (rejectionSummary.legacyUnscopedCount > 0 || falsePositiveRisk === "high") suggestedNextSteps.push("Run memory-diag rejected --verbose to inspect rejection quality.");
const wallDaysSinceActivity = daysSinceIso(model.store.lastActivityAt, now);
const dormantDiscountActive = wallDaysSinceActivity !== null && wallDaysSinceActivity > WORKSPACE_DORMANT_AFTER_DAYS;
const dormantDaysPastGrace = wallDaysSinceActivity === null ? 0 : Math.max(0, wallDaysSinceActivity - WORKSPACE_DORMANT_AFTER_DAYS);
return {
status,
summaryText: `Summary: Workspace memory quality is ${status}: ${active.length} active memories, ${evidenceCovered}/${active.length} with evidence, ${disappearances.length} evidence-only disappearances (${unknownDisappearances} unknown), ${clocks.invalid} invalid retention clocks, and ${rejectionSummary.legacyUnscopedCount} legacy unscoped rejection records.`,
active: active.length,
rendered: retention.rendered.length,
pending: model.pending.entries.length,
rejectedLast7Days,
evidenceCoveragePercent,
needsAttention,
suggestedNextSteps,
caps: {
active: active.length,
maxEntries: model.store.limits.maxEntries,
rendered: retention.rendered.length,
typeCapped: retention.typeCapped.length,
globalCapped: retention.globalCapped.length,
typeCounts,
capsFull,
},
retention: clocks,
evidence: {
currentWithEvidence: evidenceCovered,
currentWithoutEvidence: active.length - evidenceCovered,
evidenceMemoryIds: model.evidenceByMemoryId.size,
disappearances: disappearances.length,
unknownDisappearances,
withTerminalReason: disappearances.length - unknownDisappearances,
},
rejections: { ...rejectionSummary, falsePositiveRisk },
dormant: {
lastActivityAt: model.store.lastActivityAt,
wallDaysSinceActivity,
dormantDiscountActive,
dormantDaysPastGrace,
},
topRendered: retention.rendered.slice(0, 5),
weakestActive: retention.sorted.slice(-5).reverse(),
};
}
export async function runStatus(options: CliOptions): Promise<CommandResult> {
const now = Date.now();
const model = await buildInspectionReadModel(options);
const readout = buildStatusReadout(model, now);
if (options.json) {
const root = options.workspace ?? process.cwd();
const diag = memoryDiagJSONFromSnapshot(root, model.snapshot);
return {
stdout: JSON.stringify({
...diag,
summary: {
...diag.summary,
status: readout.status,
evidenceCoveragePercent: readout.evidenceCoveragePercent,
needsAttention: readout.needsAttention,
suggestedNextSteps: readout.suggestedNextSteps,
},
}, null, 2),
};
}
return {
stdout: formatStatus(readout, model, {
verbose: options.verbose,
noEmoji: options.noEmoji,
isTty: process.stdout.isTTY === true,
raw: options.raw,
}),
};
}
-21
View File
@@ -1,21 +0,0 @@
import type { LongTermType } from "../../src/types.ts";
import type { Origin } from "./types.ts";
export const TYPES: LongTermType[] = ["feedback", "decision", "project", "reference"];
export const SUSPICIOUS_REASONS = [
"progress_snapshot",
"active_file_snapshot",
"commit_or_ci_snapshot",
"temporary_status",
"raw_error",
"code_or_api_signature",
] as const;
export const ALLOWED_ORIGINS = new Set<Origin>([
"explicit_trigger",
"compaction_candidate",
"manual",
"migration_check",
"unknown",
]);
-21
View File
@@ -1,21 +0,0 @@
import type { EvidenceEventV1 } from "../../src/evidence-log.ts";
import { uniqueStrings } from "./text.ts";
export function eventMemoryIds(event: EvidenceEventV1): string[] {
return uniqueStrings([
event.memory?.memoryId ?? "",
...(event.relations ?? []).map(relation => relation.memory?.memoryId ?? ""),
]);
}
export function groupEvidenceByMemoryId(events: EvidenceEventV1[]): Map<string, EvidenceEventV1[]> {
const grouped = new Map<string, EvidenceEventV1[]>();
for (const event of events) {
for (const id of eventMemoryIds(event)) {
const bucket = grouped.get(id) ?? [];
bucket.push(event);
grouped.set(id, bucket);
}
}
return grouped;
}
-48
View File
@@ -1,48 +0,0 @@
import { cleanPath, cleanText, countBy, formatWorkspaceIdentity, sortedCounts, truncate } from "../text.ts";
import type { MigrationLogRecord } from "../types.ts";
export type MigrationAuditReport = {
migrationId: string;
path: string;
invalidLines: number;
superseded: MigrationLogRecord[];
hardReasons: string[];
risky: Array<{ record: MigrationLogRecord; reasons: string[] }>;
};
export function formatMigrationAudit(reports: MigrationAuditReport[], options: { raw: boolean }): string {
const lines: string[] = [];
lines.push("Migration audit report");
lines.push("");
if (reports.length === 0) {
lines.push("No migration logs found.");
return lines.join("\n");
}
for (let i = 0; i < reports.length; i += 1) {
const report = reports[i];
if (i > 0) lines.push("");
lines.push(`Migration: ${report.migrationId}`);
lines.push(`logPath=${cleanPath(report.path, options.raw)}`);
if (report.invalidLines > 0) lines.push(`Invalid JSONL lines skipped: ${report.invalidLines}`);
lines.push(`Superseded entries: ${report.superseded.length}`);
lines.push("");
lines.push("By hard reason:");
const byHardReason = sortedCounts(countBy(report.hardReasons));
if (byHardReason.length === 0) lines.push(" (none)");
else for (const [reason, count] of byHardReason) lines.push(` ${reason.padEnd(24)} ${count}`);
lines.push("");
lines.push("Potentially risky supersedes:");
lines.push(` ${report.risky.length}`);
for (const item of report.risky.slice(0, 10)) {
const record = item.record;
const hard = Array.isArray(record.hardReasons) ? record.hardReasons : [];
const identity = formatWorkspaceIdentity(record.workspaceKey, record.workspaceRoot, options.raw);
lines.push(` - [${record.type ?? "unknown"}] hardReasons=${JSON.stringify(hard)} risk=${item.reasons.join(",")} ${JSON.stringify(truncate(cleanText(record.text ?? "", options.raw)))}`);
if (identity) lines.push(` ${identity}`);
}
}
return lines.join("\n");
}
@@ -1,34 +0,0 @@
import type { coverageRows } from "../inspection-model.ts";
import { countBy, objectFromCounts } from "../text.ts";
import type { CoverageClass } from "../types.ts";
export type CoverageRows = ReturnType<typeof coverageRows>;
export function buildCoverageJSON(rows: CoverageRows, generatedAt = new Date().toISOString()): Record<string, unknown> {
const classCounts = objectFromCounts(countBy(rows.map(row => row.class)));
return {
version: 1,
generatedAt,
classCounts,
memories: rows,
};
}
export function formatCoverage(rows: CoverageRows): string {
const classCounts = objectFromCounts(countBy(rows.map(row => row.class)));
const lines: string[] = [];
lines.push("Memory evidence coverage");
lines.push("");
lines.push("Class counts:");
for (const cls of ["full_lifecycle", "render_only", "no_evidence", "historical_absent_with_reason", "historical_absent_unknown_reason"] as CoverageClass[]) {
lines.push(` ${cls}: ${classCounts[cls] ?? 0}`);
}
lines.push("");
lines.push("Per-memory rows:");
if (rows.length === 0) lines.push(" (none)");
for (const row of rows) {
const phases = row.eventCounts.byPhase;
lines.push(` ${row.id} ${row.class} current=${row.current ? "yes" : "no"} total=${row.eventCounts.total} extraction=${phases.extraction ?? 0} promotion=${phases.promotion ?? 0} render=${phases.render ?? 0} storage=${phases.storage ?? 0}`);
}
return lines.join("\n");
}
-42
View File
@@ -1,42 +0,0 @@
import type { EvidenceEventV1 } from "../../../src/evidence-log.ts";
import { formatStrength } from "../retention-model.ts";
import type { WorkspaceDiagSnapshot } from "../types.ts";
export function formatEvidenceRefs(eventIds: string[], allEvents: EvidenceEventV1[]): string {
if (eventIds.length === 0) return "(none)";
const byId = new Map(allEvents.map(event => [event.eventId, event]));
return eventIds
.map(id => {
const event = byId.get(id);
return event ? `${event.eventId} ${event.type}` : id;
})
.join(", ");
}
export function formatExplain(snapshot: WorkspaceDiagSnapshot): string {
const lines: string[] = [];
lines.push("Workspace memory explainability");
lines.push("");
if (snapshot.memories.length === 0) {
lines.push("No memories found.");
}
for (const memory of snapshot.memories) {
lines.push(`Memory ${memory.id}: ${memory.status}`);
const strength = typeof memory.strength === "number" ? formatStrength(memory.strength) : "n/a";
lines.push(`- strength=${strength}, type=${memory.type}, source=${memory.source}`);
lines.push(`- reasons: ${memory.reasonCodes.length > 0 ? memory.reasonCodes.join(", ") : "(none)"}`);
lines.push(`- evidence: ${formatEvidenceRefs(memory.evidenceEventIds, snapshot.allEvents)}`);
lines.push("");
}
const quarantines = snapshot.recentEvents.filter(event => event.type === "storage_corrupt_json_quarantined");
if (quarantines.length > 0) {
lines.push("Quarantined stores:");
for (const event of quarantines) {
lines.push(`- quarantined_corrupt_store: ${event.eventId} ${event.type}; reasons=${event.reasonCodes.join(",")}`);
}
}
return lines.join("\n");
}
-102
View File
@@ -1,102 +0,0 @@
import type { disappearanceRows } from "../inspection-model.ts";
import { eventCounts } from "../inspection-model.ts";
import { formatDetails } from "../text.ts";
export type DisappearanceRows = ReturnType<typeof disappearanceRows>;
export type MissingSummary = {
total: number;
explained: number;
needsReview: number;
};
export function missingSummary(rows: DisappearanceRows): MissingSummary {
return {
total: rows.length,
explained: rows.filter(row => row.classification === "historical_absent_with_reason").length,
needsReview: rows.filter(row => row.classification === "historical_absent_unknown_reason").length,
};
}
export function buildMissingJSON(rows: DisappearanceRows, options: { explain?: boolean; generatedAt?: string } = {}): Record<string, unknown> {
return {
...buildDisappearancesJSON(rows, options),
summary: missingSummary(rows),
};
}
function buildDisappearancesJSON(rows: DisappearanceRows, options: { explain?: boolean; generatedAt?: string } = {}): Record<string, unknown> {
return {
version: 1,
generatedAt: options.generatedAt ?? new Date().toISOString(),
disappearances: rows.map(row => ({
id: row.id,
classification: row.classification,
terminalType: row.terminalType,
reasonCodes: row.reasonCodes,
eventCounts: eventCounts(row.events),
details: options.explain ? row.event?.details : undefined,
})),
};
}
function formatDisappearances(rows: DisappearanceRows, options: { explain?: boolean } = {}): string {
const lines: string[] = [];
lines.push("Memory disappearances");
lines.push("");
if (rows.length === 0) {
lines.push("No evidence-only memories found.");
return lines.join("\n");
}
for (const row of rows) {
const reasons = row.reasonCodes.length > 0 ? row.reasonCodes.join(",") : "none";
lines.push(`Memory ${row.id}: ${row.classification} terminal=${row.terminalType} reasons=${reasons}`);
if (options.explain) {
lines.push(` events: ${row.events.map(event => event.type).join(", ")}`);
if (row.event?.type === "memory_removed_capacity") {
lines.push(` memory_removed_capacity details: ${formatDetails(row.event.details)}`);
}
const renderTypeCap = row.events.find(event => event.type === "render_omitted" && event.reasonCodes.includes("type_cap"));
if (renderTypeCap) {
lines.push(` render_omitted type-cap observation: reasons=${renderTypeCap.reasonCodes.join(",")} details=${formatDetails(renderTypeCap.details)}`);
}
}
}
return lines.join("\n");
}
export function formatMissing(rows: DisappearanceRows, options: { verbose?: boolean; explain?: boolean } = {}): string {
const summary = missingSummary(rows);
const lines: string[] = [];
lines.push("Missing memory summary");
lines.push("");
if (rows.length === 0) {
lines.push("No missing memories found.");
return lines.join("\n");
}
lines.push(`Total missing: ${summary.total}`);
lines.push(`Explained: ${summary.explained}`);
lines.push(`Needs review: ${summary.needsReview}`);
lines.push("");
const unknownRows = rows
.filter(row => row.classification === "historical_absent_unknown_reason")
.slice(0, 5);
lines.push("Unknown disappearance samples:");
if (unknownRows.length === 0) {
lines.push(" (none)");
} else {
for (const row of unknownRows) {
const reasons = row.reasonCodes.length > 0 ? row.reasonCodes.join(",") : "none";
lines.push(` - ${row.id} terminal=${row.terminalType} reasons=${reasons}`);
}
}
if (options.verbose || options.explain) {
lines.push("");
lines.push(formatDisappearances(rows, { explain: true }));
}
return lines.join("\n");
}
@@ -1,93 +0,0 @@
import { rejectionFalsePositiveRisk, rejectionQualitySummary } from "../rejections-model.ts";
import { cleanPath, cleanText, countBy, sortedCounts, truncate } from "../text.ts";
import type { NormalizedRejection } from "../types.ts";
export type RejectedSummary = ReturnType<typeof rejectionQualitySummary>;
export type RejectedFalsePositiveRisk = ReturnType<typeof rejectionFalsePositiveRisk>;
export function buildRejectedJSON(input: {
summary: RejectedSummary;
falsePositiveRisk: RejectedFalsePositiveRisk;
generatedAt?: string;
}): Record<string, unknown> {
return {
version: 1,
generatedAt: input.generatedAt ?? new Date().toISOString(),
...input.summary,
falsePositiveRisk: input.falsePositiveRisk,
};
}
export function formatRejected(input: {
path: string;
invalidLines: number;
records: NormalizedRejection[];
summary: RejectedSummary;
falsePositiveRisk: RejectedFalsePositiveRisk;
raw: boolean;
verbose?: boolean;
}): string {
const lines: string[] = [];
lines.push("Rejected memory summary");
lines.push("");
lines.push(`Total rejected: ${input.summary.totalRecords}`);
lines.push(`Unique texts: ${input.summary.uniqueTexts}`);
lines.push("");
lines.push("Top reasons:");
const byReason = sortedCounts(countBy(input.records.flatMap(record => record.reasons))).slice(0, 5);
if (byReason.length === 0) lines.push(" (none)");
else for (const [reason, count] of byReason) lines.push(` ${reason.padEnd(36)} ${count}`);
lines.push("");
lines.push(`False-positive risk: ${input.falsePositiveRisk}`);
if (input.falsePositiveRisk === "high") {
lines.push("⚠️ Possible false positives detected. Use --verbose to inspect samples.");
}
lines.push("");
lines.push("Recent samples:");
const samples = [...input.records]
.sort((a, b) => (new Date(b.timestamp).getTime() || 0) - (new Date(a.timestamp).getTime() || 0))
.slice(0, 5);
if (samples.length === 0) {
lines.push(" (none)");
} else {
for (const record of samples) {
lines.push(` - [${record.type}] ${truncate(cleanText(record.text, input.raw))}`);
lines.push(` reasons: ${record.reasons.length > 0 ? record.reasons.join(",") : "none"}`);
}
}
if (!input.verbose) return lines.join("\n");
lines.push("");
lines.push("Possible false-positive grouping is heuristic, not deterministic truth.");
lines.push(`logPath=${cleanPath(input.path, input.raw)}`);
if (input.invalidLines > 0) lines.push(`Invalid JSONL lines skipped: ${input.invalidLines}`);
lines.push("");
lines.push("Reason distribution (raw records):");
for (const [reason, count] of Object.entries(input.summary.reasonDistribution)) lines.push(` ${reason.padEnd(36)} ${count}`);
if (Object.keys(input.summary.reasonDistribution).length === 0) lines.push(" (none)");
lines.push("");
lines.push("Reason distribution (unique text):");
for (const [reason, count] of Object.entries(input.summary.uniqueReasonDistribution)) lines.push(` ${reason.padEnd(36)} ${count}`);
if (Object.keys(input.summary.uniqueReasonDistribution).length === 0) lines.push(" (none)");
lines.push("");
lines.push("By origin:");
const byOrigin = sortedCounts(countBy(input.records.map(record => record.origin)));
if (byOrigin.length === 0) lines.push(" (none)");
else for (const [origin, count] of byOrigin) lines.push(` ${origin.padEnd(36)} ${count}`);
lines.push("");
lines.push("Possible false-positive groups (heuristic, not deterministic):");
for (const [group, data] of Object.entries(input.summary.possibleFalsePositiveGroups)) {
lines.push(` ${group}: ${data.count}`);
for (const sample of data.samples) lines.push(` - ${JSON.stringify(cleanText(sample, input.raw))}`);
}
return lines.join("\n");
}
-155
View File
@@ -1,155 +0,0 @@
import {
DORMANT_DECAY_MULTIPLIER,
RETENTION_TYPE_MAX,
} from "../../../src/retention.ts";
import { TYPES } from "../constants.ts";
import { daysSinceIso, formatStrength } from "../retention-model.ts";
import { cleanText, truncate } from "../text.ts";
import type { MemoryInspectionReadModel, RetentionDiagItem } from "../types.ts";
export type MemoryStatusLevel = "ok" | "warning" | "degraded";
export type StatusReadout = {
status: MemoryStatusLevel;
summaryText: string;
active: number;
rendered: number;
pending: number;
rejectedLast7Days: number;
evidenceCoveragePercent: number;
needsAttention: string[];
suggestedNextSteps: string[];
caps: {
active: number;
maxEntries: number;
rendered: number;
typeCapped: number;
globalCapped: number;
typeCounts: Record<string, number>;
capsFull: boolean;
};
retention: { present: number; missing: number; invalid: number };
evidence: {
currentWithEvidence: number;
currentWithoutEvidence: number;
evidenceMemoryIds: number;
disappearances: number;
unknownDisappearances: number;
withTerminalReason: number;
};
rejections: {
totalRecords: number;
workspaceScopedCount: number;
legacyUnscopedCount: number;
falsePositiveRisk: string;
};
dormant: {
lastActivityAt?: string;
wallDaysSinceActivity: number | null;
dormantDiscountActive: boolean;
dormantDaysPastGrace: number;
};
topRendered: RetentionDiagItem[];
weakestActive: RetentionDiagItem[];
};
export function formatStatus(readout: StatusReadout, model: MemoryInspectionReadModel, options: { verbose?: boolean; noEmoji?: boolean; isTty?: boolean; raw?: boolean } = {}): string {
if (options.verbose) return formatStatusVerbose(readout, model, options);
const lines: string[] = [];
const label = statusLabel(readout.status, options);
lines.push(`${label} Memory status`);
lines.push("");
lines.push("Key metrics:");
lines.push(` active memories: ${readout.active}`);
lines.push(` rendered: ${readout.rendered}`);
lines.push(` pending: ${readout.pending}`);
lines.push(` rejected 7d: ${readout.rejectedLast7Days}`);
lines.push(` evidence coverage: ${readout.evidenceCoveragePercent}%`);
if (readout.needsAttention.length > 0) {
lines.push("");
lines.push("Needs attention:");
for (const item of readout.needsAttention) lines.push(` - ${item}`);
}
if (readout.status !== "ok" && readout.suggestedNextSteps.length > 0) {
lines.push("");
lines.push("Suggested next steps:");
for (const item of readout.suggestedNextSteps) lines.push(` - ${item}`);
}
return lines.join("\n");
}
function statusLabel(status: MemoryStatusLevel, options: { noEmoji?: boolean; isTty?: boolean }): string {
const text = status === "ok" ? "OK" : status === "warning" ? "WARNING" : "DEGRADED";
if (options.noEmoji || !options.isTty) return text;
const emoji = status === "ok" ? "🧠" : status === "warning" ? "⚠️" : "✖️";
return `${emoji} ${text}`;
}
function formatStatusVerbose(readout: StatusReadout, model: MemoryInspectionReadModel, options: { raw?: boolean } = {}): string {
const lines: string[] = [];
lines.push("Memory status inspection");
lines.push("");
lines.push(readout.summaryText);
lines.push("");
lines.push("Caps:");
lines.push(` active: ${readout.caps.active} / ${readout.caps.maxEntries}`);
for (const type of TYPES) {
const count = readout.caps.typeCounts[type] ?? 0;
const limit = RETENTION_TYPE_MAX[type];
const marker = count >= limit ? " FULL" : "";
lines.push(` ${type}: ${count} / ${limit}${marker}`);
}
lines.push(` rendered: ${readout.caps.rendered}`);
lines.push(` type-capped entries: ${readout.caps.typeCapped}`);
lines.push(` global-cap overflow: ${readout.caps.globalCapped}`);
lines.push(` caps full: ${readout.caps.capsFull ? "yes" : "no"}`);
lines.push("");
lines.push("Retention clocks:");
lines.push(` present: ${readout.retention.present}`);
lines.push(` missing: ${readout.retention.missing}`);
lines.push(` invalid: ${readout.retention.invalid}`);
lines.push("");
lines.push("Evidence:");
lines.push(` current with evidence: ${readout.evidence.currentWithEvidence}`);
lines.push(` current without evidence: ${readout.evidence.currentWithoutEvidence}`);
lines.push(` evidence memory ids: ${readout.evidence.evidenceMemoryIds}`);
lines.push(` disappearances: ${readout.evidence.disappearances}`);
lines.push(` unknown disappearances: ${readout.evidence.unknownDisappearances}`);
lines.push("");
lines.push("Rejection scoping:");
lines.push(` total records: ${readout.rejections.totalRecords}`);
lines.push(` workspace scoped: ${readout.rejections.workspaceScopedCount}`);
lines.push(` legacy unscoped: ${readout.rejections.legacyUnscopedCount}`);
lines.push(` false-positive risk: ${readout.rejections.falsePositiveRisk}`);
lines.push("");
lines.push("Dormancy:");
lines.push(` lastActivityAt: ${readout.dormant.lastActivityAt ?? "(missing)"}`);
lines.push(` wall days since activity: ${readout.dormant.wallDaysSinceActivity === null ? "unknown" : readout.dormant.wallDaysSinceActivity.toFixed(1)}`);
lines.push(` dormant discount active: ${readout.dormant.dormantDiscountActive ? "yes" : "no"}`);
lines.push(` dormant days past grace: ${readout.dormant.dormantDaysPastGrace.toFixed(1)}`);
lines.push(` dormant multiplier: ${DORMANT_DECAY_MULTIPLIER}`);
lines.push("");
lines.push("Top rendered candidates:");
pushMemoryItems(lines, readout.topRendered, options.raw === true);
lines.push("");
lines.push("Weakest active memories:");
pushMemoryItems(lines, readout.weakestActive, options.raw === true);
lines.push("");
lines.push("Store:");
lines.push(` superseded: ${model.store.entries.length - readout.active}`);
return lines.join("\n");
}
function pushMemoryItems(lines: string[], items: RetentionDiagItem[], raw: boolean): void {
if (items.length === 0) {
lines.push(" (none)");
return;
}
for (const item of items) {
lines.push(` - strength=${formatStrength(item.strength)} [${item.entry.type}] ${truncate(cleanText(item.entry.text, raw))}`);
}
}
-36
View File
@@ -1,36 +0,0 @@
import type { EvidenceEventV1 } from "../../../src/evidence-log.ts";
import { formatTraceEvent, relationMemoryIds, statusFromTraceEvent } from "../trace-model.ts";
import type { WorkspaceDiagSnapshot } from "../types.ts";
export function formatTrace(memoryId: string, snapshot: WorkspaceDiagSnapshot, trace: { events: EvidenceEventV1[] }): string {
const lines: string[] = [];
const memoryRow = snapshot.memories.find(memory => memory.id === memoryId);
const status = memoryRow?.status ?? statusFromTraceEvent(trace.events.at(-1));
lines.push(`Memory ${memoryId}: ${status}`);
lines.push("");
lines.push("Lifecycle:");
if (trace.events.length === 0) {
lines.push("(none)");
return lines.join("\n");
}
for (const event of trace.events) {
lines.push(formatTraceEvent(event));
}
const supersededBy = relationMemoryIds(trace.events, "superseded_by");
if (supersededBy.length > 0) {
lines.push("");
lines.push("Superseded by:");
for (const id of supersededBy) lines.push(`- ${id}`);
}
const reinforcedBy = relationMemoryIds(trace.events, "reinforced_by");
if (reinforcedBy.length > 0) {
lines.push("");
lines.push("Reinforced by:");
for (const id of reinforcedBy) lines.push(`- ${id}`);
}
return lines.join("\n");
}
-98
View File
@@ -1,98 +0,0 @@
import type { EvidenceEventV1 } from "../../src/evidence-log.ts";
import type { LongTermType } from "../../src/types.ts";
import { countBy, objectFromCounts, uniqueStrings } from "./text.ts";
import { groupEvidenceByMemoryId } from "./evidence-model.ts";
import { loadRejectionRecords } from "./rejections-model.ts";
import { snapshotForOptions } from "./workspace-snapshot.ts";
import type { CliOptions, CoverageClass, DisappearanceReason, MemoryInspectionReadModel } from "./types.ts";
export async function buildInspectionReadModel(options: CliOptions): Promise<MemoryInspectionReadModel> {
const snapshot = await snapshotForOptions(options);
const rejections = await loadRejectionRecords(options);
return {
snapshot,
store: snapshot.store,
pending: snapshot.journal,
evidenceEvents: snapshot.allEvents,
rejectionRecords: rejections.records,
currentById: new Map(snapshot.store.entries.map(entry => [entry.id, entry])),
evidenceByMemoryId: groupEvidenceByMemoryId(snapshot.allEvents),
};
}
export function terminalDisappearanceReason(events: EvidenceEventV1[]): DisappearanceReason {
const terminal = [...events].reverse().find(event =>
event.type === "memory_removed_capacity"
|| event.type === "promotion_absorbed_exact"
|| event.type === "promotion_absorbed_identity"
|| event.type === "promotion_superseded"
);
const renderOmission = [...events].reverse().find(event => event.type === "render_omitted");
const event = terminal ?? renderOmission;
if (!event) {
return { classification: "historical_absent_unknown_reason", terminalType: "unknown", reasonCodes: [] };
}
return {
classification: "historical_absent_with_reason",
terminalType: event.type,
reasonCodes: event.reasonCodes,
event,
};
}
export function coverageClassForMemory(id: string, model: MemoryInspectionReadModel): CoverageClass {
const events = model.evidenceByMemoryId.get(id) ?? [];
if (!model.currentById.has(id)) return terminalDisappearanceReason(events).classification;
if (events.length === 0) return "no_evidence";
if (events.every(event => event.phase === "render")) return "render_only";
return "full_lifecycle";
}
export function eventCounts(events: EvidenceEventV1[]): { total: number; byType: Record<string, number>; byPhase: Record<string, number> } {
return {
total: events.length,
byType: objectFromCounts(countBy(events.map(event => event.type))),
byPhase: objectFromCounts(countBy(events.map(event => event.phase))),
};
}
export function coverageRows(model: MemoryInspectionReadModel, includeHistorical: boolean): Array<{
id: string;
class: CoverageClass;
current: boolean;
type?: LongTermType;
eventCounts: ReturnType<typeof eventCounts>;
}> {
const ids = new Set<string>(model.currentById.keys());
if (includeHistorical) {
for (const id of model.evidenceByMemoryId.keys()) ids.add(id);
}
return [...ids].sort().map(id => {
const entry = model.currentById.get(id);
const events = model.evidenceByMemoryId.get(id) ?? [];
return {
id,
class: coverageClassForMemory(id, model),
current: Boolean(entry),
type: entry?.type ?? events.find(event => event.memory?.memoryId === id)?.memory?.type,
eventCounts: eventCounts(events),
};
});
}
export function disappearanceRows(model: MemoryInspectionReadModel): Array<{
id: string;
classification: DisappearanceReason["classification"];
terminalType: DisappearanceReason["terminalType"];
reasonCodes: string[];
event?: EvidenceEventV1;
events: EvidenceEventV1[];
}> {
const rows = [...model.evidenceByMemoryId.entries()]
.filter(([id]) => !model.currentById.has(id))
.map(([id, events]) => {
const reason = terminalDisappearanceReason(events);
return { id, ...reason, events };
});
return rows.sort((a, b) => a.classification.localeCompare(b.classification) || a.id.localeCompare(b.id));
}
-36
View File
@@ -1,36 +0,0 @@
import { existsSync } from "node:fs";
import { readFile } from "node:fs/promises";
export async function readJSONFile<T>(path: string): Promise<T | null> {
try {
return JSON.parse(await readFile(path, "utf8")) as T;
} catch {
return null;
}
}
export async function readJSONLFile<T>(path: string): Promise<{ records: T[]; invalidLines: number }> {
let content = "";
try {
content = await readFile(path, "utf8");
} catch {
return { records: [], invalidLines: 0 };
}
const records: T[] = [];
let invalidLines = 0;
for (const line of content.split("\n")) {
const trimmed = line.trim();
if (!trimmed) continue;
try {
records.push(JSON.parse(trimmed) as T);
} catch {
invalidLines += 1;
}
}
return { records, invalidLines };
}
export function pathExists(path: string): boolean {
return existsSync(path);
}
-42
View File
@@ -1,42 +0,0 @@
import { readdir } from "node:fs/promises";
import { join } from "node:path";
import { dataHome, migrationLogPath } from "../../src/paths.ts";
import type { CliOptions, MigrationLogRecord } from "./types.ts";
export function migrationLogsRoot(): string {
return join(dataHome(), "opencode-working-memory", "migration-logs");
}
export async function migrationLogPaths(options: CliOptions): Promise<string[]> {
if (options.migration) return [migrationLogPath(options.migration)];
const root = migrationLogsRoot();
let entries: string[] = [];
try {
entries = await readdir(root);
} catch {
return [];
}
return entries.filter(entry => entry.endsWith(".jsonl")).sort().map(entry => join(root, entry));
}
export function migrationIdFromPath(path: string): string {
return path.split("/").pop()?.replace(/\.jsonl$/, "") ?? "unknown";
}
export function riskySupersedeReasons(record: MigrationLogRecord): string[] {
const reasons: string[] = [];
const hardReasonsMissing = !Array.isArray(record.hardReasons);
const hardReasons = Array.isArray(record.hardReasons) ? record.hardReasons : [];
const qualityReasons = Array.isArray(record.reasons) ? record.reasons : [];
const text = record.text ?? "";
if (hardReasonsMissing || hardReasons.length === 0) reasons.push("missing_or_empty_hardReasons");
if (qualityReasons.length > 0 && hardReasons.length === 0) reasons.push("soft_reasons_without_hardReasons");
if (/\b(?:User|user|prefers|requires|wants|insists)\b|用戶|使用者|偏好|要求|不要|不刪除/u.test(text)) reasons.push("user_preference_marker");
if (/\b(?:must|should|do not|never|is|are|follows)\b|必須|應該|採用|維持|需支援/iu.test(text)) reasons.push("durable_rule_marker");
if ((record.type === "feedback" || record.type === "decision") && hardReasons.length === 1 && hardReasons[0] === "path_heavy") {
reasons.push("feedback_or_decision_path_heavy_only");
}
return reasons;
}
-136
View File
@@ -1,136 +0,0 @@
import { extractionRejectionLogPath, workspaceKey } from "../../src/paths.ts";
import { HARD_QUALITY_REASONS, isArchitectureLikeDecision } from "../../src/memory-quality.ts";
import { ALLOWED_ORIGINS } from "./constants.ts";
import { readJSONLFile } from "./io.ts";
import { canonicalMemoryText, cleanText, countBy, objectFromCounts, truncate } from "./text.ts";
import { CliInputError, type CliOptions, type NormalizedRejection, type Origin, type RejectionLogRecord } from "./types.ts";
export { CliInputError } from "./types.ts";
export function inferOrigin(record: RejectionLogRecord): Origin {
if (record.origin && ALLOWED_ORIGINS.has(record.origin as Origin)) return record.origin as Origin;
if (record.source === "compaction") return "compaction_candidate";
if (record.source === "explicit") return "explicit_trigger";
if (record.source === "manual") return "manual";
return "unknown";
}
export function normalizeRejection(record: RejectionLogRecord): NormalizedRejection | null {
if (!record.text || !Array.isArray(record.reasons)) return null;
const origin = inferOrigin(record);
return {
timestamp: record.timestamp ?? "",
workspaceKey: record.workspaceKey,
workspaceRoot: record.workspaceRoot,
workspaceRootHash: record.workspaceRootHash,
type: record.type ?? "project",
source: record.source,
origin,
fromTrigger: typeof record.fromTrigger === "boolean" ? record.fromTrigger : origin === "explicit_trigger",
text: record.text,
reasons: record.reasons,
};
}
export function sinceCutoff(rawSince: string | undefined, now = Date.now()): number | null {
if (!rawSince) return null;
const relative = rawSince.match(/^(\d+)([dhm])$/i);
if (relative) {
const amount = Number(relative[1]);
const unit = relative[2].toLowerCase();
const multiplier = unit === "d" ? 86_400_000 : unit === "h" ? 3_600_000 : 60_000;
return now - amount * multiplier;
}
const timestamp = new Date(rawSince).getTime();
if (Number.isNaN(timestamp)) throw new CliInputError(`Invalid --since value: ${rawSince}`);
return timestamp;
}
export function hasSoftReason(record: NormalizedRejection): boolean {
return record.reasons.some(reason => !HARD_QUALITY_REASONS.has(reason));
}
export function hasWorkspaceScope(record: NormalizedRejection): boolean {
return Boolean(record.workspaceKey || record.workspaceRoot || record.workspaceRootHash);
}
export async function workspaceKeyForOption(options: CliOptions): Promise<string | undefined> {
return options.workspace ? workspaceKey(options.workspace) : undefined;
}
export async function loadRejectionRecords(options: CliOptions): Promise<{ path: string; invalidLines: number; records: NormalizedRejection[] }> {
const path = extractionRejectionLogPath();
const { records, invalidLines } = await readJSONLFile<RejectionLogRecord>(path);
const cutoff = sinceCutoff(options.since);
const requestedWorkspaceKey = await workspaceKeyForOption(options);
let normalized = records.map(normalizeRejection).filter((record): record is NormalizedRejection => record !== null);
if (requestedWorkspaceKey) {
normalized = normalized.filter(record => !record.workspaceKey || record.workspaceKey === requestedWorkspaceKey);
}
if (cutoff !== null) {
normalized = normalized.filter(record => {
const timestamp = new Date(record.timestamp).getTime();
return !Number.isNaN(timestamp) && timestamp >= cutoff;
});
}
if (options.softOnly) normalized = normalized.filter(hasSoftReason);
if (options.triggerOnly) normalized = normalized.filter(record => record.fromTrigger || record.origin === "explicit_trigger");
if (options.reason) normalized = normalized.filter(record => record.reasons.includes(options.reason ?? ""));
return { path, invalidLines, records: normalized };
}
export function uniqueByCanonicalText(records: NormalizedRejection[]): NormalizedRejection[] {
const byText = new Map<string, NormalizedRejection>();
for (const record of records) {
const key = `${record.type}:${canonicalMemoryText(record.text)}`;
if (!byText.has(key)) byText.set(key, record);
}
return [...byText.values()];
}
export function rejectionQualitySummary(records: NormalizedRejection[]): {
totalRecords: number;
uniqueTexts: number;
workspaceScopedCount: number;
legacyUnscopedCount: number;
reasonDistribution: Record<string, number>;
uniqueReasonDistribution: Record<string, number>;
possibleFalsePositiveGroups: Record<string, { count: number; samples: string[] }>;
} {
const uniqueRecords = uniqueByCanonicalText(records);
const badDecisionUnique = uniqueRecords.filter(record => record.reasons.includes("bad_decision"));
const groups: Record<string, { count: number; samples: string[] }> = {
architecture_like_possible_false_positive: { count: 0, samples: [] },
clearly_garbage: { count: 0, samples: [] },
ambiguous: { count: 0, samples: [] },
};
for (const record of badDecisionUnique) {
const hardReasons = record.reasons.filter(reason => HARD_QUALITY_REASONS.has(reason));
const statusLike = /\b(?:implemented|added|updated|fixed|completed|reviewed|tests?|CI|commit|wave|phase|task|session)\b/i.test(record.text);
const group = isArchitectureLikeDecision(record.text) && hardReasons.length === 0 && !statusLike
? "architecture_like_possible_false_positive"
: hardReasons.length > 0 || statusLike
? "clearly_garbage"
: "ambiguous";
groups[group].count += 1;
if (groups[group].samples.length < 5) groups[group].samples.push(truncate(cleanText(record.text, false), 120));
}
return {
totalRecords: records.length,
uniqueTexts: uniqueRecords.length,
workspaceScopedCount: records.filter(hasWorkspaceScope).length,
legacyUnscopedCount: records.filter(record => !hasWorkspaceScope(record)).length,
reasonDistribution: objectFromCounts(countBy(records.flatMap(record => record.reasons))),
uniqueReasonDistribution: objectFromCounts(countBy(uniqueRecords.flatMap(record => record.reasons))),
possibleFalsePositiveGroups: groups,
};
}
export function rejectionFalsePositiveRisk(summary: ReturnType<typeof rejectionQualitySummary>): "low" | "high" {
const possible = summary.possibleFalsePositiveGroups.architecture_like_possible_false_positive.count;
return possible >= 3 || (summary.uniqueTexts > 0 && possible / summary.uniqueTexts >= 0.5) ? "high" : "low";
}
-84
View File
@@ -1,84 +0,0 @@
import {
RETENTION_TYPE_MAX,
calculateRetentionStrength,
} from "../../src/retention.ts";
import type { LongTermMemoryEntry, LongTermSource, LongTermType, WorkspaceMemoryStore } from "../../src/types.ts";
import { LONG_TERM_LIMITS, PROMOTION_RETRY_LIMITS } from "../../src/types.ts";
import type { RetentionDiagItem } from "./types.ts";
export function ageDays(entry: LongTermMemoryEntry, now = Date.now()): number | null {
const time = new Date(entry.createdAt).getTime();
if (Number.isNaN(time)) return null;
return Math.floor((now - time) / 86_400_000);
}
export function formatStrength(value: number): string {
return Number.isFinite(value) ? value.toFixed(3) : "0.000";
}
export function daysSinceIso(value: string | undefined, now = Date.now()): number | null {
if (!value) return null;
const ms = new Date(value).getTime();
if (!Number.isFinite(ms)) return null;
return Math.max(0, (now - ms) / 86_400_000);
}
export function isSafetyCriticalForDiag(entry: LongTermMemoryEntry): boolean {
return entry.safetyCritical === true;
}
export function retentionCandidatesForDiag(store: WorkspaceMemoryStore, now = Date.now()): {
sorted: RetentionDiagItem[];
rendered: RetentionDiagItem[];
typeCapped: RetentionDiagItem[];
globalCapped: RetentionDiagItem[];
} {
const active = store.entries.filter(entry => entry.status !== "superseded");
const sorted = active
.map(entry => ({ entry, strength: calculateRetentionStrength(entry, now, store.lastActivityAt) }))
.sort((a, b) => b.strength - a.strength || a.entry.id.localeCompare(b.entry.id));
const rendered: RetentionDiagItem[] = [];
const typeCapped: RetentionDiagItem[] = [];
const globalCapped: RetentionDiagItem[] = [];
const typeCounts: Partial<Record<LongTermType, number>> = {};
for (const item of sorted) {
const count = typeCounts[item.entry.type] ?? 0;
const max = RETENTION_TYPE_MAX[item.entry.type] ?? Infinity;
if (count >= max) {
typeCapped.push(item);
continue;
}
typeCounts[item.entry.type] = count + 1;
if (rendered.length < LONG_TERM_LIMITS.maxEntries) {
rendered.push(item);
} else {
globalCapped.push(item);
}
}
return { sorted, rendered, typeCapped, globalCapped };
}
export function promotionLimit(source: LongTermSource): number {
if (source === "manual") return PROMOTION_RETRY_LIMITS.maxManualAttempts;
return PROMOTION_RETRY_LIMITS.maxExplicitAttempts;
}
export function retentionClockSummary(entries: LongTermMemoryEntry[]): { present: number; missing: number; invalid: number } {
let present = 0;
let missing = 0;
let invalid = 0;
for (const entry of entries) {
if (entry.retentionClock === undefined) {
missing += 1;
} else if (!Number.isFinite(entry.retentionClock) || entry.retentionClock <= 0) {
invalid += 1;
} else {
present += 1;
}
}
return { present, missing, invalid };
}
-69
View File
@@ -1,69 +0,0 @@
import { createHash } from "node:crypto";
import type { EvidenceEventV1 } from "../../src/evidence-log.ts";
import { redactCredentials } from "../../src/redaction.ts";
export function countBy<T extends string>(items: T[]): Map<T, number> {
const counts = new Map<T, number>();
for (const item of items) counts.set(item, (counts.get(item) ?? 0) + 1);
return counts;
}
export function sortedCounts<T extends string>(counts: Map<T, number>): Array<[T, number]> {
return [...counts.entries()].sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0]));
}
export function workspaceRootHash(root: string): string {
return createHash("sha256").update(root).digest("hex").slice(0, 16);
}
export function redactAbsolutePaths(text: string): string {
return text.replace(/(?:^|[\s"'`(=:\[])(\/(?:Users|home|private|tmp|var|opt|Volumes|[^\s"'`)\],;:]+)\/[^\s"'`)\],;:]*)/g, (match, path) => match.replace(path, "<path>"));
}
export function cleanText(text: string, raw: boolean): string {
if (raw) return text;
return redactAbsolutePaths(redactCredentials(text));
}
export function cleanPath(path: string, raw: boolean): string {
return raw ? path : "<path>";
}
export function formatWorkspaceIdentity(workspaceKeyValue: string | undefined, workspaceRoot: string | undefined, raw: boolean): string {
const parts: string[] = [];
if (workspaceKeyValue) parts.push(`workspaceKey=${workspaceKeyValue}`);
if (workspaceRoot) {
parts.push(raw ? `workspaceRoot=${workspaceRoot}` : `workspaceRootHash=${workspaceRootHash(workspaceRoot)}`);
}
return parts.join(" ");
}
export function truncate(text: string, max = 120): string {
const collapsed = text.replace(/\s+/g, " ").trim();
return collapsed.length <= max ? collapsed : `${collapsed.slice(0, max - 1)}…`;
}
export function canonicalMemoryText(text: string): string {
return text
.normalize("NFKC")
.toLowerCase()
.replace(/[\s\p{P}]+/gu, " ")
.trim();
}
export function formatPercent(ratio: number): string {
return `${(ratio * 100).toFixed(1)}%`;
}
export function uniqueStrings(values: string[]): string[] {
return [...new Set(values.filter(Boolean))];
}
export function objectFromCounts<T extends string>(counts: Map<T, number>): Record<string, number> {
return Object.fromEntries(sortedCounts(counts));
}
export function formatDetails(details: EvidenceEventV1["details"]): string {
if (!details || Object.keys(details).length === 0) return "none";
return Object.entries(details).map(([key, value]) => `${key}=${Array.isArray(value) ? value.join("|") : String(value)}`).join(" ");
}
-30
View File
@@ -1,30 +0,0 @@
import type { EvidenceEventV1 } from "../../src/evidence-log.ts";
import { uniqueStrings } from "./text.ts";
import { statusFromOmissionReason } from "./workspace-snapshot.ts";
export function statusFromTraceEvent(event: EvidenceEventV1 | undefined): string {
if (!event) return "unknown";
if (event.type === "render_selected") return "rendered";
if (event.type === "render_omitted") return statusFromOmissionReason(event.reasonCodes[0]);
if (event.type === "promotion_absorbed_exact" || event.type === "promotion_absorbed_identity") return "omitted_absorbed_duplicate";
if (event.type === "promotion_retry_scheduled") return "pending_retry";
if (event.type === "promotion_rejected_capacity" || event.type === "promotion_retry_exhausted") return "pending_rejected_capacity";
if (event.type === "storage_corrupt_json_quarantined") return "quarantined_corrupt_store";
if (event.outcome === "superseded") return "omitted_superseded";
return event.outcome;
}
export function formatTraceEvent(event: EvidenceEventV1): string {
const reasons = event.reasonCodes.length > 0 ? event.reasonCodes.join(",") : "none";
const relations = (event.relations ?? [])
.map(relation => relation.memory?.memoryId ? `${relation.role}=${relation.memory.memoryId}` : undefined)
.filter((value): value is string => Boolean(value));
const relationText = relations.length > 0 ? `; ${relations.join(", ")}` : "";
return `- ${event.eventId} ${event.type}: ${event.outcome}; reasons=${reasons}${relationText}`;
}
export function relationMemoryIds(events: EvidenceEventV1[], role: string): string[] {
return uniqueStrings(events.flatMap(event => (event.relations ?? [])
.filter(relation => relation.role === role)
.map(relation => relation.memory?.memoryId ?? "")));
}
-157
View File
@@ -1,157 +0,0 @@
import type { EvidenceEventType, EvidenceEventV1, EvidenceOutcome } from "../../src/evidence-log.ts";
import type { LongTermMemoryEntry, LongTermSource, LongTermType, PendingMemoryJournalStore, WorkspaceMemoryStore } from "../../src/types.ts";
import type { Command } from "./command-metadata.ts";
export type { Command, HiddenCommand, VisibleCommand } from "./command-metadata.ts";
export type MemoryRenderStatus =
| "rendered"
| "omitted_superseded"
| "omitted_type_cap"
| "omitted_global_cap"
| "omitted_char_budget"
| "omitted_absorbed_duplicate"
| "pending_retry"
| "pending_rejected_capacity"
| "quarantined_corrupt_store";
export type MemoryDiagJSON = {
version: 1;
workspace: { rootHash: string; key: string };
generatedAt: string;
summary: {
storedActive: number;
rendered: number;
pending: number;
rejectedLast7Days: number;
corruptStoresQuarantinedLast30Days: number;
status?: "ok" | "warning" | "degraded";
evidenceCoveragePercent?: number;
needsAttention?: string[];
suggestedNextSteps?: string[];
};
memories: Array<{
id: string;
type: "feedback" | "project" | "decision" | "reference";
source: "explicit" | "compaction" | "manual";
status: MemoryRenderStatus;
strength?: number;
reasonCodes: string[];
textPreview?: string;
evidenceEventIds: string[];
}>;
recentEvents: Array<{
eventId: string;
type: EvidenceEventType;
outcome: EvidenceOutcome;
createdAt: string;
memoryId?: string;
reasonCodes: string[];
}>;
};
export type Origin = "explicit_trigger" | "compaction_candidate" | "manual" | "migration_check" | "unknown";
export type CliOptions = {
raw: boolean;
json?: boolean;
verbose?: boolean;
noEmoji?: boolean;
workspace?: string;
all?: boolean;
softOnly?: boolean;
triggerOnly?: boolean;
includeHistorical?: boolean;
reason?: string;
explain?: boolean;
since?: string;
migration?: string;
memory?: string;
positional?: string[];
auditMode?: "coverage" | "migrations";
};
export type ParsedArgs =
| { ok: true; command: Command; options: CliOptions; deprecationNotice?: string }
| { ok: true; help: true; usage: string }
| { ok: false; message: string; usage: string; exitCode: number };
export type CommandResult = { stdout: string; stderr?: string; exitCode?: number };
export class CliInputError extends Error {}
export type RejectionLogRecord = {
timestamp?: string;
workspaceKey?: string;
workspaceRoot?: string;
workspaceRootHash?: string;
type?: LongTermType;
source?: LongTermSource | string;
origin?: string;
fromTrigger?: boolean;
text?: string;
reasons?: string[];
};
export type NormalizedRejection = Required<Pick<RejectionLogRecord, "timestamp" | "type" | "text" | "reasons">> & {
workspaceKey?: string;
workspaceRoot?: string;
workspaceRootHash?: string;
source?: string;
origin: Origin;
fromTrigger: boolean;
};
export type MigrationLogRecord = {
migrationId?: string;
timestamp?: string;
workspaceKey?: string;
workspaceRoot?: string;
entryId?: string;
type?: LongTermType;
source?: LongTermSource | string;
text?: string;
reasons?: string[];
hardReasons?: string[];
beforeStatus?: string;
afterStatus?: string;
};
export type RetentionDiagItem = {
entry: LongTermMemoryEntry;
strength: number;
};
export type WorkspaceDiagSnapshot = {
store: WorkspaceMemoryStore;
journal: PendingMemoryJournalStore;
retention: {
sorted: RetentionDiagItem[];
rendered: RetentionDiagItem[];
typeCapped: RetentionDiagItem[];
globalCapped: RetentionDiagItem[];
};
memories: MemoryDiagJSON["memories"];
recentEvents: MemoryDiagJSON["recentEvents"];
allEvents: EvidenceEventV1[];
summary: MemoryDiagJSON["summary"];
};
export type MemoryInspectionReadModel = {
snapshot: WorkspaceDiagSnapshot;
store: WorkspaceMemoryStore;
pending: PendingMemoryJournalStore;
evidenceEvents: EvidenceEventV1[];
rejectionRecords: NormalizedRejection[];
currentById: Map<string, LongTermMemoryEntry>;
evidenceByMemoryId: Map<string, EvidenceEventV1[]>;
};
export type CoverageClass = "full_lifecycle" | "render_only" | "no_evidence" | "historical_absent_with_reason" | "historical_absent_unknown_reason";
export type DisappearanceReason = {
classification: "historical_absent_with_reason" | "historical_absent_unknown_reason";
terminalType: EvidenceEventType | "unknown";
reasonCodes: string[];
event?: EvidenceEventV1;
};
-236
View File
@@ -1,236 +0,0 @@
import { calculateRetentionStrength } from "../../src/retention.ts";
import {
queryEvidenceEvents,
type EvidenceEventV1,
} from "../../src/evidence-log.ts";
import { workspaceKey, workspaceMemoryPath, workspacePendingJournalPath } from "../../src/paths.ts";
import { accountWorkspaceMemoryRender } from "../../src/workspace-memory.ts";
import type { LongTermMemoryEntry, PendingMemoryJournalStore, WorkspaceMemoryStore } from "../../src/types.ts";
import { LONG_TERM_LIMITS } from "../../src/types.ts";
import { readJSONFile } from "./io.ts";
import { groupEvidenceByMemoryId } from "./evidence-model.ts";
import { promotionLimit, retentionCandidatesForDiag } from "./retention-model.ts";
import { cleanText, truncate, uniqueStrings, workspaceRootHash } from "./text.ts";
import type { CliOptions, MemoryDiagJSON, MemoryRenderStatus, WorkspaceDiagSnapshot } from "./types.ts";
export function emptyStore(root: string, key: string): WorkspaceMemoryStore {
return {
version: 1,
workspace: { root, key },
limits: { maxRenderedChars: LONG_TERM_LIMITS.maxRenderedChars, maxEntries: LONG_TERM_LIMITS.maxEntries },
entries: [],
migrations: [],
updatedAt: new Date(0).toISOString(),
};
}
export function normalizedStore(store: WorkspaceMemoryStore | null, root: string, key: string): WorkspaceMemoryStore {
const fallback = emptyStore(root, key);
return {
...fallback,
...(store ?? {}),
workspace: store?.workspace ?? fallback.workspace,
limits: {
maxRenderedChars: store?.limits?.maxRenderedChars ?? fallback.limits.maxRenderedChars,
maxEntries: store?.limits?.maxEntries ?? fallback.limits.maxEntries,
},
entries: Array.isArray(store?.entries) ? store.entries : [],
migrations: Array.isArray(store?.migrations) ? store.migrations : [],
};
}
export function normalizedJournal(journal: PendingMemoryJournalStore | null): PendingMemoryJournalStore {
return {
version: 1,
workspace: journal?.workspace ?? { root: "", key: "" },
entries: Array.isArray(journal?.entries) ? journal.entries : [],
updatedAt: journal?.updatedAt ?? new Date(0).toISOString(),
};
}
export function eventMemoryId(event: EvidenceEventV1): string | undefined {
return event.memory?.memoryId
?? event.relations?.map(relation => relation.memory?.memoryId).find((id): id is string => Boolean(id));
}
export function isWithinDays(iso: string, days: number, now = Date.now()): boolean {
const ms = new Date(iso).getTime();
return Number.isFinite(ms) && ms >= now - days * 86_400_000;
}
export function renderStatusReason(status: MemoryRenderStatus, fallback?: string): string[] {
switch (status) {
case "rendered": return ["within_caps", "within_char_budget"];
case "omitted_superseded": return ["superseded"];
case "omitted_type_cap": return ["type_cap"];
case "omitted_global_cap": return ["global_cap"];
case "omitted_char_budget": return [fallback === "empty_render_budget" ? "empty_render_budget" : "char_budget"];
case "omitted_absorbed_duplicate": return ["absorbed_duplicate"];
case "pending_retry": return ["retryable_capacity_rejection"];
case "pending_rejected_capacity": return ["capacity_rejected", "max_attempts_reached"];
case "quarantined_corrupt_store": return ["invalid_json"];
}
}
export function statusFromOmissionReason(reason: string | undefined): MemoryRenderStatus {
if (reason === "superseded") return "omitted_superseded";
if (reason === "type_cap") return "omitted_type_cap";
if (reason === "global_cap") return "omitted_global_cap";
return "omitted_char_budget";
}
export function pendingStatus(entry: LongTermMemoryEntry): MemoryRenderStatus {
const attempts = entry.promotionAttempts ?? 0;
return attempts >= promotionLimit(entry.source) ? "pending_rejected_capacity" : "pending_retry";
}
export function safeTextPreview(text: string): string {
return truncate(cleanText(text, false), 120);
}
function evidenceSummaryForMemory(grouped: Map<string, EvidenceEventV1[]>, memoryId: string): { eventIds: string[]; reasonCodes: string[] } {
const events = grouped.get(memoryId) ?? [];
const reasonCodes = new Set<string>();
for (const event of events) {
for (const reason of event.reasonCodes) reasonCodes.add(reason);
}
return {
eventIds: events.map(event => event.eventId),
reasonCodes: [...reasonCodes],
};
}
export async function buildWorkspaceDiagSnapshot(input: {
root: string;
key: string;
memoryPath: string;
pendingPath: string;
}, now = Date.now()): Promise<WorkspaceDiagSnapshot> {
const rawStore = await readJSONFile<WorkspaceMemoryStore>(input.memoryPath);
const storeRoot = rawStore?.workspace?.root ?? input.root;
const storeKey = rawStore?.workspace?.key ?? input.key;
const store = normalizedStore(rawStore, storeRoot, storeKey);
const journal = normalizedJournal(await readJSONFile<PendingMemoryJournalStore>(input.pendingPath));
const retention = retentionCandidatesForDiag(store, now);
const renderAccounting = accountWorkspaceMemoryRender(store);
const renderedIds = new Set(renderAccounting.rendered.map(memory => memory.id));
const omittedById = new Map(renderAccounting.omitted.map(item => [item.memory.id, item.reason]));
const allEvents = await queryEvidenceEvents(input.root);
const evidenceByMemoryId = groupEvidenceByMemoryId(allEvents);
const recentEvidence = await queryEvidenceEvents(input.root, { newestFirst: true, limit: 50 });
const memoryRows: MemoryDiagJSON["memories"] = [];
const seenIds = new Set<string>();
for (const entry of store.entries) {
const omissionReason = omittedById.get(entry.id);
const status: MemoryRenderStatus = renderedIds.has(entry.id)
? "rendered"
: statusFromOmissionReason(omissionReason ?? (entry.status === "superseded" ? "superseded" : undefined));
const summary = evidenceSummaryForMemory(evidenceByMemoryId, entry.id);
memoryRows.push({
id: entry.id,
type: entry.type,
source: entry.source,
status,
strength: calculateRetentionStrength(entry, now, store.lastActivityAt),
reasonCodes: uniqueStrings([...renderStatusReason(status, omissionReason), ...summary.reasonCodes]),
textPreview: safeTextPreview(entry.text),
evidenceEventIds: summary.eventIds,
});
seenIds.add(entry.id);
}
for (const entry of journal.entries) {
const status = pendingStatus(entry);
const summary = evidenceSummaryForMemory(evidenceByMemoryId, entry.id);
memoryRows.push({
id: entry.id,
type: entry.type,
source: entry.source,
status,
strength: calculateRetentionStrength(entry, now, store.lastActivityAt),
reasonCodes: uniqueStrings([...renderStatusReason(status), entry.lastPromotionFailureReason ?? "", ...summary.reasonCodes]),
textPreview: safeTextPreview(entry.text),
evidenceEventIds: summary.eventIds,
});
seenIds.add(entry.id);
}
for (const event of allEvents) {
if (event.outcome !== "absorbed") continue;
const memory = event.memory;
if (!memory?.memoryId || !memory.type || !memory.source || seenIds.has(memory.memoryId)) continue;
const summary = evidenceSummaryForMemory(evidenceByMemoryId, memory.memoryId);
memoryRows.push({
id: memory.memoryId,
type: memory.type,
source: memory.source,
status: "omitted_absorbed_duplicate",
reasonCodes: uniqueStrings([...renderStatusReason("omitted_absorbed_duplicate"), ...summary.reasonCodes]),
evidenceEventIds: summary.eventIds.length > 0 ? summary.eventIds : [event.eventId],
});
seenIds.add(memory.memoryId);
}
const recentEvents = recentEvidence.map(event => ({
eventId: event.eventId,
type: event.type,
outcome: event.outcome,
createdAt: event.createdAt,
memoryId: eventMemoryId(event),
reasonCodes: uniqueStrings([
...event.reasonCodes,
...(event.type === "storage_corrupt_json_quarantined" ? ["quarantined_corrupt_store"] : []),
]),
}));
return {
store,
journal,
retention,
memories: memoryRows,
recentEvents,
allEvents,
summary: {
storedActive: store.entries.filter(entry => entry.status !== "superseded").length,
rendered: retention.rendered.length,
pending: journal.entries.length,
rejectedLast7Days: allEvents.filter(event => event.outcome === "rejected" && isWithinDays(event.createdAt, 7, now)).length,
corruptStoresQuarantinedLast30Days: allEvents.filter(event => event.type === "storage_corrupt_json_quarantined" && isWithinDays(event.createdAt, 30, now)).length,
},
};
}
export async function buildMemoryDiagJSON(root: string): Promise<MemoryDiagJSON> {
const key = await workspaceKey(root);
const snapshot = await buildWorkspaceDiagSnapshot({
root,
key,
memoryPath: await workspaceMemoryPath(root),
pendingPath: await workspacePendingJournalPath(root),
});
return memoryDiagJSONFromSnapshot(root, snapshot);
}
export function memoryDiagJSONFromSnapshot(root: string, snapshot: WorkspaceDiagSnapshot, generatedAt = new Date().toISOString()): MemoryDiagJSON {
return {
version: 1,
workspace: { rootHash: workspaceRootHash(snapshot.store.workspace.root || root), key: snapshot.store.workspace.key },
generatedAt,
summary: snapshot.summary,
memories: snapshot.memories,
recentEvents: snapshot.recentEvents,
};
}
export async function snapshotForOptions(options: CliOptions): Promise<WorkspaceDiagSnapshot> {
const root = options.workspace ?? process.cwd();
const key = await workspaceKey(root);
return buildWorkspaceDiagSnapshot({
root,
key,
memoryPath: await workspaceMemoryPath(root),
pendingPath: await workspacePendingJournalPath(root),
});
}
-530
View File
@@ -1,530 +0,0 @@
import { createHash } from "node:crypto";
import { existsSync } from "node:fs";
import { appendFile, mkdir, readFile, realpath, rename, rm, stat, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { dataHome, workspaceEvidenceLogPath, workspaceKey } from "./paths.ts";
import { redactCredentials } from "./redaction.ts";
export type EvidenceEventType =
| "extraction_candidate_accepted"
| "extraction_candidate_rejected"
| "explicit_memory_detected"
| "explicit_memory_ignored"
| "pending_memory_appended"
| "pending_memory_cleared"
| "promotion_promoted"
| "promotion_absorbed_exact"
| "promotion_absorbed_identity"
| "promotion_superseded"
| "promotion_rejected_capacity"
| "promotion_retry_scheduled"
| "promotion_retry_exhausted"
| "memory_reinforced"
| "memory_migration_superseded"
| "render_selected"
| "render_omitted"
| "memory_removed_capacity"
| "storage_corrupt_json_quarantined"
| "storage_stale_lock_recovered"
| "storage_lock_timeout"
| "hook_failed";
export type EvidencePhase =
| "extraction"
| "explicit"
| "pending_journal"
| "promotion"
| "reinforcement"
| "render"
| "storage"
| "hook";
export type EvidenceOutcome =
| "accepted"
| "rejected"
| "promoted"
| "absorbed"
| "superseded"
| "rendered"
| "omitted"
| "removed"
| "retried"
| "exhausted"
| "reinforced"
| "quarantined"
| "failed"
| "recovered";
export type MemoryEvidenceRef = {
memoryId?: string;
memoryKeyHash?: string;
identityKeyHash?: string;
type?: "feedback" | "project" | "decision" | "reference";
source?: "explicit" | "compaction" | "manual";
status?: "active" | "superseded";
};
export type EvidenceRelation = {
role:
| "candidate"
| "pending"
| "promoted"
| "retained"
| "absorbed"
| "superseded"
| "superseded_by"
| "reinforced"
| "reinforced_by"
| "rendered"
| "omitted"
| "removed";
memory?: MemoryEvidenceRef;
};
export type EvidenceDetailValue = string | number | boolean | null | string[] | number[];
export type EvidenceEventV1 = {
version: 1;
eventId: string;
createdAt: string;
workspaceKey: string;
workspaceRootHash: string;
sessionHash?: string;
messageHash?: string;
type: EvidenceEventType;
phase: EvidencePhase;
outcome: EvidenceOutcome;
memory?: MemoryEvidenceRef;
relations?: EvidenceRelation[];
reasonCodes: string[];
details?: Record<string, EvidenceDetailValue>;
textPreview?: string;
};
export type EvidenceEventInput = Omit<
EvidenceEventV1,
"version" | "eventId" | "createdAt" | "workspaceKey" | "workspaceRootHash"
>;
export type EvidenceQuery = {
since?: string;
until?: string;
types?: EvidenceEventType[];
phases?: EvidencePhase[];
outcomes?: EvidenceOutcome[];
memoryId?: string;
memoryKeyHash?: string;
identityKeyHash?: string;
sessionHash?: string;
limit?: number;
newestFirst?: boolean;
};
export type MemoryEvidenceSummary = {
memoryId?: string;
memoryKeyHash?: string;
latestOutcome?: EvidenceOutcome;
latestRenderStatus?: "rendered" | "omitted";
reasonCodes: string[];
eventIds: string[];
lastEventAt?: string;
};
export type MemoryLifecycleTrace = {
memoryId?: string;
memoryKeyHash?: string;
identityKeyHash?: string;
events: EvidenceEventV1[];
createdBy?: EvidenceEventV1;
acceptedBy?: EvidenceEventV1;
promotedBy?: EvidenceEventV1;
absorbedBy?: EvidenceEventV1;
supersededBy?: EvidenceEventV1;
reinforcedBy: EvidenceEventV1[];
latestRender?: EvidenceEventV1;
currentStatus:
| "accepted"
| "pending"
| "promoted"
| "absorbed"
| "superseded"
| "rendered"
| "omitted"
| "rejected"
| "unknown";
};
export const EVIDENCE_LOG_LIMITS = {
maxAgeDays: 90,
maxEventsPerWorkspace: 5000,
maxBytesPerWorkspace: 2 * 1024 * 1024,
pruneEveryAppendCount: 100,
} as const;
const appendCounts = new Map<string, number>();
const HASH_PATTERN = /^[a-f0-9]{16}$/i;
const DAY_MS = 24 * 60 * 60 * 1000;
const MAX_DETAIL_STRING_CHARS = 240;
const MAX_DETAIL_ARRAY_ITEMS = 25;
function evidenceHash(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 16);
}
function normalizeHashValue(value: string | undefined): string | undefined {
if (!value) return undefined;
return HASH_PATTERN.test(value) ? value.toLowerCase() : evidenceHash(value);
}
async function resolvedRoot(root: string): Promise<string> {
return realpath(root).catch(() => root);
}
function evidenceTextPreview(text: string, maxChars = 120): string {
return redactCredentials(text).replace(/\s+/g, " ").trim().slice(0, maxChars);
}
function sanitizeReasonCode(reason: string): string {
return reason.replace(/[^a-zA-Z0-9_.:-]/g, "_").slice(0, 120);
}
function sanitizeMemoryRef(memory: MemoryEvidenceRef | undefined): MemoryEvidenceRef | undefined {
if (!memory) return undefined;
const sanitized: MemoryEvidenceRef = {};
if (typeof memory.memoryId === "string" && memory.memoryId) sanitized.memoryId = memory.memoryId.slice(0, 160);
if (memory.memoryKeyHash) sanitized.memoryKeyHash = normalizeHashValue(memory.memoryKeyHash);
if (memory.identityKeyHash) sanitized.identityKeyHash = normalizeHashValue(memory.identityKeyHash);
if (memory.type) sanitized.type = memory.type;
if (memory.source) sanitized.source = memory.source;
if (memory.status) sanitized.status = memory.status;
return Object.keys(sanitized).length > 0 ? sanitized : undefined;
}
function sanitizeRelations(relations: EvidenceRelation[] | undefined): EvidenceRelation[] | undefined {
if (!relations) return undefined;
const sanitized = relations
.map(relation => ({
role: relation.role,
memory: sanitizeMemoryRef(relation.memory),
}))
.slice(0, 25);
return sanitized.length > 0 ? sanitized : undefined;
}
function sanitizeDetailString(value: string): string {
return evidenceTextPreview(value, MAX_DETAIL_STRING_CHARS);
}
function sanitizeDetails(details: EvidenceEventInput["details"]): EvidenceEventV1["details"] {
if (!details) return undefined;
const sanitized: Record<string, EvidenceDetailValue> = {};
for (const [rawKey, rawValue] of Object.entries(details).slice(0, 50)) {
const key = rawKey.replace(/[^a-zA-Z0-9_.:-]/g, "_").slice(0, 80);
if (!key) continue;
if (typeof rawValue === "string") {
sanitized[key] = sanitizeDetailString(rawValue);
} else if (typeof rawValue === "number") {
if (Number.isFinite(rawValue)) sanitized[key] = rawValue;
} else if (typeof rawValue === "boolean" || rawValue === null) {
sanitized[key] = rawValue;
} else if (Array.isArray(rawValue)) {
if (rawValue.every(item => typeof item === "string")) {
sanitized[key] = rawValue.slice(0, MAX_DETAIL_ARRAY_ITEMS).map(item => sanitizeDetailString(item));
} else if (rawValue.every(item => typeof item === "number" && Number.isFinite(item))) {
sanitized[key] = rawValue.slice(0, MAX_DETAIL_ARRAY_ITEMS) as number[];
}
}
}
return Object.keys(sanitized).length > 0 ? sanitized : undefined;
}
function buildEvidenceEvent(
input: EvidenceEventInput,
workspaceKeyValue: string,
workspaceRootHash: string,
): EvidenceEventV1 {
const textPreviewMax = input.type === "extraction_candidate_rejected" ? 80 : 120;
const event: EvidenceEventV1 = {
version: 1,
eventId: `evt_${Date.now()}_${Math.random().toString(36).slice(2, 10).padEnd(8, "0")}`,
createdAt: new Date().toISOString(),
workspaceKey: workspaceKeyValue,
workspaceRootHash,
type: input.type,
phase: input.phase,
outcome: input.outcome,
reasonCodes: input.reasonCodes.map(sanitizeReasonCode).filter(Boolean).slice(0, 25),
};
const memory = sanitizeMemoryRef(input.memory);
const relations = sanitizeRelations(input.relations);
const details = sanitizeDetails(input.details);
if (input.sessionHash) event.sessionHash = normalizeHashValue(input.sessionHash);
if (input.messageHash) event.messageHash = normalizeHashValue(input.messageHash);
if (memory) event.memory = memory;
if (relations) event.relations = relations;
if (details) event.details = details;
if (input.textPreview) event.textPreview = evidenceTextPreview(input.textPreview, textPreviewMax);
return event;
}
async function safeAppendEvidenceLine(path: string, line: string): Promise<void> {
try {
await mkdir(dirname(path), { recursive: true });
await appendFile(path, `${line}\n`, "utf8");
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
console.error(`[memory] failed to write evidence event: ${message}`);
}
}
async function maybePruneEvidenceLog(path: string): Promise<void> {
const nextCount = (appendCounts.get(path) ?? 0) + 1;
appendCounts.set(path, nextCount);
if (nextCount % EVIDENCE_LOG_LIMITS.pruneEveryAppendCount !== 0) return;
try {
await pruneEvidenceLogPath(path);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
console.error(`[memory] failed to prune evidence log: ${message}`);
}
}
export async function appendEvidenceEvent(root: string, event: EvidenceEventInput): Promise<EvidenceEventV1> {
const records = await appendEvidenceEvents(root, [event]);
return records[0];
}
export async function appendEvidenceEvents(root: string, events: EvidenceEventInput[]): Promise<EvidenceEventV1[]> {
const path = await workspaceEvidenceLogPath(root);
const rootPath = await resolvedRoot(root);
const workspaceRootHash = evidenceHash(rootPath);
const workspaceKeyValue = await workspaceKey(root);
const records = events.map(event => buildEvidenceEvent(event, workspaceKeyValue, workspaceRootHash));
for (const record of records) {
await safeAppendEvidenceLine(path, JSON.stringify(record));
await maybePruneEvidenceLog(path);
}
return records;
}
export async function appendEvidenceEventForWorkspaceKey(
workspaceKeyValue: string,
event: EvidenceEventInput,
): Promise<EvidenceEventV1> {
const path = join(dataHome(), "opencode-working-memory", "workspaces", workspaceKeyValue, "evidence", "events.jsonl");
const record = buildEvidenceEvent(event, workspaceKeyValue, workspaceKeyValue);
await safeAppendEvidenceLine(path, JSON.stringify(record));
await maybePruneEvidenceLog(path);
return record;
}
type ParsedEvidenceLine = {
event: EvidenceEventV1;
index: number;
};
function parseEvidenceLine(line: string): EvidenceEventV1 | null {
try {
const parsed = JSON.parse(line) as Partial<EvidenceEventV1>;
if (parsed.version !== 1 || !parsed.eventId || !parsed.createdAt || !parsed.type) return null;
return parsed as EvidenceEventV1;
} catch {
return null;
}
}
async function readEvidenceLines(path: string, warnInvalid: boolean): Promise<{ valid: ParsedEvidenceLine[]; invalid: string[] }> {
if (!existsSync(path)) return { valid: [], invalid: [] };
const raw = await readFile(path, "utf8");
const valid: ParsedEvidenceLine[] = [];
const invalid: string[] = [];
raw.split(/\n/).forEach((line, index) => {
if (!line.trim()) return;
const event = parseEvidenceLine(line);
if (event) {
valid.push({ event, index });
} else {
invalid.push(line);
if (warnInvalid) console.warn(`[memory] skipped invalid evidence log line ${index + 1}`);
}
});
return { valid, invalid };
}
function eventTimeMs(event: EvidenceEventV1): number {
const ms = new Date(event.createdAt).getTime();
return Number.isFinite(ms) ? ms : 0;
}
async function atomicWriteText(path: string, text: string): Promise<void> {
await mkdir(dirname(path), { recursive: true });
const tmp = `${path}.${process.pid}.${Date.now()}.${Math.random().toString(36).slice(2, 10)}.tmp`;
try {
await writeFile(tmp, text, { encoding: "utf8", mode: 0o600 });
await rename(tmp, path);
} catch (error) {
await rm(tmp, { force: true }).catch(() => undefined);
throw error;
}
}
function serializeEvents(events: EvidenceEventV1[]): string {
return events.map(event => JSON.stringify(event)).join("\n") + (events.length > 0 ? "\n" : "");
}
function trimEventsToByteLimit(events: EvidenceEventV1[]): EvidenceEventV1[] {
let kept = [...events];
while (kept.length > 0 && Buffer.byteLength(serializeEvents(kept), "utf8") > EVIDENCE_LOG_LIMITS.maxBytesPerWorkspace) {
kept = kept.slice(1);
}
return kept;
}
async function pruneEvidenceLogPath(path: string): Promise<void> {
if (!existsSync(path)) return;
const stats = await stat(path);
if (stats.isDirectory()) return;
const { valid, invalid } = await readEvidenceLines(path, false);
if (invalid.length > 0) {
const corruptPath = `${path}.corrupt-lines-${Date.now()}.jsonl`;
await writeFile(corruptPath, invalid.join("\n") + "\n", { encoding: "utf8", mode: 0o600 }).catch(error => {
const message = error instanceof Error ? error.message : String(error);
console.error(`[memory] failed to quarantine invalid evidence lines: ${message}`);
});
}
const cutoff = Date.now() - EVIDENCE_LOG_LIMITS.maxAgeDays * DAY_MS;
let events = valid
.filter(item => eventTimeMs(item.event) >= cutoff)
.sort((a, b) => eventTimeMs(a.event) - eventTimeMs(b.event) || a.index - b.index)
.map(item => item.event);
if (events.length > EVIDENCE_LOG_LIMITS.maxEventsPerWorkspace) {
events = events.slice(events.length - EVIDENCE_LOG_LIMITS.maxEventsPerWorkspace);
}
events = trimEventsToByteLimit(events);
await atomicWriteText(path, serializeEvents(events));
}
function memoryRefMatches(memory: MemoryEvidenceRef | undefined, query: Pick<EvidenceQuery, "memoryId" | "memoryKeyHash" | "identityKeyHash">): boolean {
if (!memory) return false;
const memoryKeyHash = normalizeHashValue(query.memoryKeyHash);
const identityKeyHash = normalizeHashValue(query.identityKeyHash);
if (query.memoryId && memory.memoryId === query.memoryId) return true;
if (memoryKeyHash && memory.memoryKeyHash === memoryKeyHash) return true;
if (identityKeyHash && memory.identityKeyHash === identityKeyHash) return true;
return false;
}
function eventMatchesMemory(event: EvidenceEventV1, query: Pick<EvidenceQuery, "memoryId" | "memoryKeyHash" | "identityKeyHash">): boolean {
if (!query.memoryId && !query.memoryKeyHash && !query.identityKeyHash) return true;
if (memoryRefMatches(event.memory, query)) return true;
return (event.relations ?? []).some(relation => memoryRefMatches(relation.memory, query));
}
export async function queryEvidenceEvents(
root: string,
query: EvidenceQuery = {},
): Promise<EvidenceEventV1[]> {
const path = await workspaceEvidenceLogPath(root);
const { valid } = await readEvidenceLines(path, true);
const sinceMs = query.since ? new Date(query.since).getTime() : undefined;
const untilMs = query.until ? new Date(query.until).getTime() : undefined;
const sessionHash = normalizeHashValue(query.sessionHash);
let events = valid.map(item => item.event).filter(event => {
const createdAtMs = eventTimeMs(event);
if (Number.isFinite(sinceMs) && createdAtMs < sinceMs) return false;
if (Number.isFinite(untilMs) && createdAtMs > untilMs) return false;
if (query.types && !query.types.includes(event.type)) return false;
if (query.phases && !query.phases.includes(event.phase)) return false;
if (query.outcomes && !query.outcomes.includes(event.outcome)) return false;
if (sessionHash && event.sessionHash !== sessionHash) return false;
if (!eventMatchesMemory(event, query)) return false;
return true;
});
if (query.newestFirst) events = events.slice().reverse();
if (typeof query.limit === "number" && query.limit >= 0) events = events.slice(0, query.limit);
return events;
}
export async function summarizeMemoryEvidence(
root: string,
input: { memoryId?: string; memoryKeyHash?: string },
): Promise<MemoryEvidenceSummary> {
const events = await queryEvidenceEvents(root, input);
const latest = events.at(-1);
const latestRender = events.filter(event => event.phase === "render").at(-1);
const reasonCodes = new Set<string>();
for (const event of events) {
for (const reason of event.reasonCodes) reasonCodes.add(reason);
}
return {
memoryId: input.memoryId,
memoryKeyHash: normalizeHashValue(input.memoryKeyHash),
latestOutcome: latest?.outcome,
latestRenderStatus: latestRender?.outcome === "rendered" || latestRender?.outcome === "omitted"
? latestRender.outcome
: undefined,
reasonCodes: [...reasonCodes],
eventIds: events.map(event => event.eventId),
lastEventAt: latest?.createdAt,
};
}
function currentStatusFromEvent(event: EvidenceEventV1 | undefined): MemoryLifecycleTrace["currentStatus"] {
if (!event) return "unknown";
if (event.outcome === "accepted") return "accepted";
if (event.outcome === "promoted") return "promoted";
if (event.outcome === "absorbed") return "absorbed";
if (event.outcome === "superseded") return "superseded";
if (event.outcome === "rendered") return "rendered";
if (event.outcome === "omitted") return "omitted";
if (event.outcome === "rejected") return "rejected";
if (event.type === "pending_memory_appended") return "pending";
return "unknown";
}
export async function traceMemoryLifecycle(
root: string,
input: { memoryId?: string; memoryKeyHash?: string; identityKeyHash?: string },
): Promise<MemoryLifecycleTrace> {
const events = await queryEvidenceEvents(root, input);
const createdBy = events.find(event => event.type === "explicit_memory_detected" || event.type === "pending_memory_appended");
const acceptedBy = events.find(event => event.type === "extraction_candidate_accepted" || event.type === "explicit_memory_detected");
const promotedBy = events.find(event => event.type === "promotion_promoted");
const absorbedBy = events.find(event => event.outcome === "absorbed");
const supersededBy = events.find(event => event.outcome === "superseded");
const reinforcedBy = events.filter(event => event.type === "memory_reinforced" || event.outcome === "reinforced");
const latestRender = events.filter(event => event.phase === "render").at(-1);
const latest = events.at(-1);
return {
memoryId: input.memoryId,
memoryKeyHash: normalizeHashValue(input.memoryKeyHash),
identityKeyHash: normalizeHashValue(input.identityKeyHash),
events,
createdBy,
acceptedBy,
promotedBy,
absorbedBy,
supersededBy,
reinforcedBy,
latestRender,
currentStatus: currentStatusFromEvent(latest),
};
}
+29 -192
View File
@@ -6,7 +6,6 @@ import { LONG_TERM_LIMITS } from "./types.ts";
import { assessMemoryQuality } from "./memory-quality.ts";
import { extractionRejectionLogPath } from "./paths.ts";
import { redactCredentials } from "./redaction.ts";
import type { EvidenceEventInput } from "./evidence-log.ts";
function id(prefix: string): string {
return `${prefix}_${Date.now()}_${Math.random().toString(36).slice(2, 8)}`;
@@ -47,34 +46,6 @@ function isNegatedMemoryRequest(text: string, matchIndex: number): boolean {
}
export function extractExplicitMemories(text: string): LongTermMemoryEntry[] {
return extractExplicitMemoriesWithEvidence(text).entries;
}
export type WorkspaceMemoryParseResult = {
entries: LongTermMemoryEntry[];
evidence: EvidenceEventInput[];
};
function evidenceTextPreview(text: string, maxChars = 120): string {
return redactCredentials(text).replace(/\s+/g, " ").trim().slice(0, maxChars);
}
function memoryEvidence(memory: LongTermMemoryEntry): EvidenceEventInput["memory"] {
return {
memoryId: memory.id,
type: memory.type,
source: memory.source,
status: memory.status,
};
}
function extractionEvidence(
input: Pick<EvidenceEventInput, "type" | "phase" | "outcome" | "reasonCodes" | "textPreview" | "memory" | "details">,
): EvidenceEventInput {
return input;
}
export function extractExplicitMemoriesWithEvidence(text: string): WorkspaceMemoryParseResult {
// 注意:所有pattern必須有 g flag,因為使用 matchAll()
// Pattern 必須在行首匹配,避免匹配到句子中間的非指令式用法
const patterns = [
@@ -97,77 +68,31 @@ export function extractExplicitMemoriesWithEvidence(text: string): WorkspaceMemo
/(?:^|\n)\s*(?:my preference is|i prefer)[::,,]?\s*(.+)$/gim,
];
const nowMs = Date.now();
const now = new Date(nowMs).toISOString();
const now = new Date().toISOString();
const entries: LongTermMemoryEntry[] = [];
const evidence: EvidenceEventInput[] = [];
const seen = new Set<string>();
const negatedLinePattern = /(?:^|\n)\s*(?:(?:please\s+)?(?:do\s+not|don't|dont|never)\s+remember(?:\s+(?:this|that))?|不要\s*(?:記住|记住)|別\s*(?:記住|记住)|别\s*(?:記住|记住))[::,,]?\s*(.+)$/gim;
for (const match of text.matchAll(negatedLinePattern)) {
evidence.push(extractionEvidence({
type: "explicit_memory_ignored",
phase: "explicit",
outcome: "rejected",
reasonCodes: ["negated_request"],
textPreview: evidenceTextPreview(match[1] ?? match[0], 80),
}));
}
for (const pattern of patterns) {
for (const match of text.matchAll(pattern)) {
const body = match[1]?.trim();
if (body && /^(再说|再說|later|next time)$/i.test(body)) {
evidence.push(extractionEvidence({
type: "explicit_memory_ignored",
phase: "explicit",
outcome: "rejected",
reasonCodes: ["deferral"],
textPreview: evidenceTextPreview(body, 80),
}));
continue;
}
if (!body || body.length < 8) {
evidence.push(extractionEvidence({
type: "explicit_memory_ignored",
phase: "explicit",
outcome: "rejected",
reasonCodes: ["too_short"],
textPreview: evidenceTextPreview(body ?? match[0], 80),
}));
continue;
}
if (!body || body.length < 8) continue;
// Calculate actual trigger position (after possible newline)
const triggerIndex = match.index! + (match[0].match(/^[\s\n]*/)?.[0]?.length || 0);
// Check if this is a negated request (e.g., "不要記住")
if (isNegatedMemoryRequest(text, triggerIndex)) {
evidence.push(extractionEvidence({
type: "explicit_memory_ignored",
phase: "explicit",
outcome: "rejected",
reasonCodes: ["negated_request"],
textPreview: evidenceTextPreview(body, 80),
}));
continue;
}
if (isNegatedMemoryRequest(text, triggerIndex)) continue;
// Check if it's a deferral (e.g., "later", "next time")
if (/^(再说|再說|later|next time)$/i.test(body)) continue;
// Dedupe by canonical body
const key = body.toLowerCase().replace(/\s+/g, " ").trim();
if (seen.has(key)) {
evidence.push(extractionEvidence({
type: "explicit_memory_ignored",
phase: "explicit",
outcome: "rejected",
reasonCodes: ["duplicate_in_message"],
textPreview: evidenceTextPreview(body, 80),
}));
continue;
}
if (seen.has(key)) continue;
seen.add(key);
const type = classifyExplicitMemory(body);
const memory: LongTermMemoryEntry = {
entries.push({
id: id("mem"),
type,
text: body.slice(0, LONG_TERM_LIMITS.maxEntryTextChars),
@@ -176,22 +101,12 @@ export function extractExplicitMemoriesWithEvidence(text: string): WorkspaceMemo
status: "active",
createdAt: now,
updatedAt: now,
retentionClock: nowMs,
staleAfterDays: staleAfterDaysFor(type),
};
entries.push(memory);
evidence.push(extractionEvidence({
type: "explicit_memory_detected",
phase: "explicit",
outcome: "accepted",
reasonCodes: ["explicit_trigger_matched"],
memory: memoryEvidence(memory),
textPreview: evidenceTextPreview(memory.text),
}));
});
}
}
return { entries, evidence };
return entries;
}
function classifyExplicitMemory(text: string): LongTermType {
@@ -322,13 +237,6 @@ type ExtractionRejectionLogEntry = {
text: string;
reasons: string[];
source: "compaction";
workspaceKey?: string;
workspaceRootHash?: string;
};
type WorkspaceMemoryCandidateParseOptions = {
workspaceKey?: string;
workspaceRootHash?: string;
};
async function logExtractionRejection(entry: ExtractionRejectionLogEntry): Promise<void> {
@@ -341,15 +249,15 @@ async function logExtractionRejection(entry: ExtractionRejectionLogEntry): Promi
}
}
function evaluateWorkspaceMemoryCandidate(
function shouldAcceptWorkspaceMemoryCandidate(
entry: {
type: LongTermType;
text: string;
},
options: {
fromMemoryTrigger?: boolean;
} & WorkspaceMemoryCandidateParseOptions = {},
): { accepted: boolean; reasons: string[] } {
} = {},
): boolean {
const text = entry.text.trim();
const minLength = options.fromMemoryTrigger ? 6 : 20;
@@ -357,14 +265,14 @@ function evaluateWorkspaceMemoryCandidate(
if (entry.type === "reference" && /\b(?:admin\s+)?pin\s|scrypt|n=\d+|r=\d+|p=\d+/i.test(text)) {
// Stable config values can be short — allow below generic min length
} else if (text.length < minLength) {
return { accepted: false, reasons: ["too_short"] };
return false;
}
// Indirect Prompt Injection / Adversarial Instructions
// Rejects attempts to overwrite system behavior or "ignore" rules.
// comparative "instead of" is allowed.
if (/\b(ignore\s+all|ignore\s+previous|ignore\s+instruction|overwrite\s+system|overwrite\s+rules|forget\s+all|delete\s+root)\b/i.test(text)) return { accepted: false, reasons: ["prompt_injection"] };
if (/\b(ignore|instruction|overwrite)\b/i.test(text) && /\b(previous|all|rules|behavior|prompt|system)\b/i.test(text)) return { accepted: false, reasons: ["prompt_injection"] };
if (/\b(ignore\s+all|ignore\s+previous|ignore\s+instruction|overwrite\s+system|overwrite\s+rules|forget\s+all|delete\s+root)\b/i.test(text)) return false;
if (/\b(ignore|instruction|overwrite)\b/i.test(text) && /\b(previous|all|rules|behavior|prompt|system)\b/i.test(text)) return false;
const quality = assessMemoryQuality({ type: entry.type, text, source: "compaction" });
if (!quality.accepted) {
@@ -374,25 +282,11 @@ function evaluateWorkspaceMemoryCandidate(
text: redactCredentials(text),
reasons: quality.reasons,
source: "compaction",
workspaceKey: options.workspaceKey,
workspaceRootHash: options.workspaceRootHash,
});
return { accepted: false, reasons: quality.reasons };
return false;
}
return { accepted: true, reasons: ["quality_gate_passed"] };
}
function shouldAcceptWorkspaceMemoryCandidate(
entry: {
type: LongTermType;
text: string;
},
options: {
fromMemoryTrigger?: boolean;
} & WorkspaceMemoryCandidateParseOptions = {},
): boolean {
return evaluateWorkspaceMemoryCandidate(entry, options).accepted;
return true;
}
/**
@@ -419,24 +313,12 @@ function extractCandidateBlock(summary: string): string | null {
return null;
}
export function parseWorkspaceMemoryCandidates(
summary: string,
options?: WorkspaceMemoryCandidateParseOptions,
): LongTermMemoryEntry[] {
return parseWorkspaceMemoryCandidatesWithEvidence(summary, options).entries;
}
export function parseWorkspaceMemoryCandidatesWithEvidence(
summary: string,
options: WorkspaceMemoryCandidateParseOptions = {},
): WorkspaceMemoryParseResult {
export function parseWorkspaceMemoryCandidates(summary: string): LongTermMemoryEntry[] {
const block = extractCandidateBlock(summary);
if (!block) return { entries: [], evidence: [] };
if (!block) return [];
const nowMs = Date.now();
const now = new Date(nowMs).toISOString();
const now = new Date().toISOString();
const entries: LongTermMemoryEntry[] = [];
const evidence: EvidenceEventInput[] = [];
for (const line of block.split("\n")) {
// Accept both "- [type] text" (bracketed) and "- type text" (bracketless)
@@ -446,53 +328,18 @@ export function parseWorkspaceMemoryCandidatesWithEvidence(
if (!item) continue;
const type = (item[1] ?? item[2]).toLowerCase() as LongTermType;
const normalizedBody = normalizeCandidateBody(item[3]);
if (!normalizedBody) {
evidence.push(extractionEvidence({
type: "extraction_candidate_rejected",
phase: "extraction",
outcome: "rejected",
reasonCodes: ["negated_request"],
memory: { type, source: "compaction" },
textPreview: evidenceTextPreview(item[3], 80),
}));
continue;
}
if (!normalizedBody) continue;
const minLength = normalizedBody.hadTrigger ? 6 : 12;
if (normalizedBody.text.length < minLength) {
evidence.push(extractionEvidence({
type: "extraction_candidate_rejected",
phase: "extraction",
outcome: "rejected",
reasonCodes: ["too_short"],
memory: { type, source: "compaction" },
textPreview: evidenceTextPreview(normalizedBody.text, 80),
}));
continue;
}
if (normalizedBody.text.length < minLength) continue;
// Apply quality gate
const quality = evaluateWorkspaceMemoryCandidate(
if (!shouldAcceptWorkspaceMemoryCandidate(
{ type, text: normalizedBody.text },
{
fromMemoryTrigger: normalizedBody.hadTrigger,
workspaceKey: options.workspaceKey,
workspaceRootHash: options.workspaceRootHash,
},
);
if (!quality.accepted) {
evidence.push(extractionEvidence({
type: "extraction_candidate_rejected",
phase: "extraction",
outcome: "rejected",
reasonCodes: quality.reasons,
memory: { type, source: "compaction" },
textPreview: evidenceTextPreview(normalizedBody.text, 80),
}));
continue;
}
{ fromMemoryTrigger: normalizedBody.hadTrigger },
)) continue;
const memory: LongTermMemoryEntry = {
entries.push({
id: id("mem"),
type,
text: normalizedBody.text.slice(0, LONG_TERM_LIMITS.maxEntryTextChars),
@@ -501,19 +348,9 @@ export function parseWorkspaceMemoryCandidatesWithEvidence(
status: "active",
createdAt: now,
updatedAt: now,
retentionClock: nowMs,
staleAfterDays: staleAfterDaysFor(type),
};
entries.push(memory);
evidence.push(extractionEvidence({
type: "extraction_candidate_accepted",
phase: "extraction",
outcome: "accepted",
reasonCodes: ["quality_gate_passed", "valid_candidate_format"],
memory: memoryEvidence(memory),
textPreview: evidenceTextPreview(memory.text),
}));
});
}
return { entries, evidence };
return entries;
}
+3 -27
View File
@@ -78,39 +78,15 @@ export function isFeedbackQualityViolation(text: string): boolean {
return true;
}
export function hasFutureRule(text: string): boolean {
return /\b(?:use|keep|prefer|avoid|do not|don't|must|should|never|always|require|choose|reject)\b/i.test(text)
export function isDecisionQualityViolation(text: string): boolean {
const futureRule = /\b(?:use|keep|prefer|avoid|do not|don't|must|should|never|always|require|choose|reject)\b/i.test(text)
|| /(?:使用|保持|避免|不要|必須|必须|應該|应该|選擇|选择)/u.test(text);
}
export function isArchitectureLikeDecision(text: string): boolean {
if (/\b(?:[A-Z][A-Z0-9]*_[A-Z0-9_]*|[A-Z][A-Z0-9]{3,})\b/.test(text)) return true;
if (/\b(?:schema|model|scoring|retention|cap|evidence|normalization|root cause|architecture(?!\s+keywords?)|boundary|rule|memory system)\b/i.test(text)) return true;
if (/(?:模型|架構|架构|证据|證據|規則|规则|邊界|边界|記憶系統|记忆系统|原因|採用|采用)/u.test(text)) return true;
return false;
}
export function isImplementationStatusDecision(text: string): boolean {
if (!futureRule) return true;
if (/\b(?:implemented|added|updated|fixed|completed|reviewed)\b/i.test(text)) return true;
if (/\b(?:was|were|has been|had been)\b/i.test(text) && /\b(?:previous|last|latest|this session|this wave|already)\b/i.test(text)) return true;
return false;
}
export function isDecisionQualityViolation(text: string): boolean {
if (hasFutureRule(text)) {
if (isImplementationStatusDecision(text)) return true;
return false;
}
if (isArchitectureLikeDecision(text)) {
if (isImplementationStatusDecision(text)) return true;
if (/\b(?:session|wave|task|test|CI|compatibility|commit)\b/i.test(text)) return true;
return false;
}
return true;
}
function isRawErrorViolation(text: string): boolean {
if (/^\s*(Error|TypeError|ReferenceError|SyntaxError|Exception):/i.test(text)) return true;
if (/at \S+ \([^)]+:\d+:\d+\)/.test(text)) return true;
-4
View File
@@ -24,10 +24,6 @@ export async function workspacePendingJournalPath(root: string): Promise<string>
return join(await memoryRoot(root), "workspace-pending-journal.json");
}
export async function workspaceEvidenceLogPath(root: string): Promise<string> {
return join(await memoryRoot(root), "evidence", "events.jsonl");
}
export async function sessionStatePath(root: string, sessionID: string): Promise<string> {
const safeSessionID = createHash("sha256").update(sessionID).digest("hex").slice(0, 32);
return join(await memoryRoot(root), "sessions", `${safeSessionID}.json`);
+143 -293
View File
@@ -23,23 +23,20 @@
* - Parses compaction summaries for memory candidates and merges them
*/
import { createHash } from "node:crypto";
import { realpath, rm } from "fs/promises";
import { rm } from "fs/promises";
import type { Plugin } from "@opencode-ai/plugin";
import {
extractExplicitMemoriesWithEvidence,
extractExplicitMemories,
extractActiveFiles,
extractErrorsFromBash,
parseWorkspaceMemoryCandidatesWithEvidence,
parseWorkspaceMemoryCandidates,
} from "./extractors.ts";
import {
loadWorkspaceMemory,
updateWorkspaceMemory,
updateWorkspaceMemoryWithAccounting,
accountWorkspaceMemoryRender,
workspaceMemoryIdentityKey,
renderWorkspaceMemory,
} from "./workspace-memory.ts";
import { reinforceMemory } from "./retention.ts";
import {
appendPendingMemories,
clearPendingMemories,
@@ -58,15 +55,14 @@ import {
addRecentDecision,
renderHotSessionState,
} from "./session-state.ts";
import { sessionStatePath, workspaceKey } from "./paths.ts";
import { sessionStatePath } from "./paths.ts";
import {
latestUserText,
latestCompactionSummary,
pendingTodos,
} from "./opencode.ts";
import { accountPendingPromotions, promotionAccountingEvidenceEvents } from "./promotion-accounting.ts";
import { appendEvidenceEvent, appendEvidenceEvents, type EvidenceEventInput, type MemoryEvidenceRef } from "./evidence-log.ts";
import { type LongTermMemoryEntry, WORKSPACE_MEMORY_CACHE_LIMITS } from "./types.ts";
import { accountPendingPromotions } from "./promotion-accounting.ts";
import { WORKSPACE_MEMORY_CACHE_LIMITS } from "./types.ts";
/**
* Build the complete compaction prompt.
@@ -116,9 +112,6 @@ function buildCompactionPrompt(privateContext: string): string {
"",
"CRITICAL MEMORY RULES:",
"- Most compactions should produce ZERO memories. Empty is correct when nothing durable changed.",
"- Existing workspace memory may already contain durable facts. If a fact is already present and still accurate, do not create a rephrased duplicate.",
"- If the same durable fact truly needs to be emitted again, reuse the existing memory wording exactly whenever possible.",
"- Only emit a new memory when the fact is new, materially corrected, or materially more specific than the existing memory.",
"- NO completion or progress statements: do not extract completed work, passing tests, commits, PR status, wave/task/phase completion, or current state.",
"- NO session-internal implementation notes: do not extract what files were edited, what bug was just fixed, what command just ran, or what the assistant reviewed.",
"- feedback ONLY means stable user preferences or user instructions, written in imperative/future-facing form.",
@@ -168,38 +161,6 @@ function renderTodosForCompaction(todos: Array<{ content: string; status: string
return lines.join("\n");
}
function safeErrorMessage(error: unknown): string {
const message = error instanceof Error ? error.message : String(error);
return message.replace(/\s+/g, " ").slice(0, 240);
}
async function warnMemoryHook(scope: string, error: unknown, root?: string): Promise<void> {
const message = safeErrorMessage(error);
console.error(`[memory] ${scope} failed: ${message}`);
if (root) {
await appendEvidenceEvent(root, {
type: "hook_failed",
phase: "hook",
outcome: "failed",
reasonCodes: [scope],
details: { message },
}).catch(() => undefined);
}
}
async function workspaceRootHash(root: string): Promise<string> {
const resolved = await realpath(root).catch(() => root);
return createHash("sha256").update(resolved).digest("hex").slice(0, 16);
}
async function workspaceIdentity(root: string): Promise<{ workspaceKey: string; workspaceRootHash: string }> {
const [workspaceKeyValue, workspaceRootHashValue] = await Promise.all([
workspaceKey(root),
workspaceRootHash(root),
]);
return { workspaceKey: workspaceKeyValue, workspaceRootHash: workspaceRootHashValue };
}
export const MemoryV2Plugin: Plugin = async (input) => {
const { directory, client } = input;
@@ -236,29 +197,6 @@ export const MemoryV2Plugin: Plugin = async (input) => {
// Cache for processed user message IDs (to avoid duplicate processing)
const processedUserMessages = new Map<string, Set<string>>();
function memoryEvidenceRef(memory: LongTermMemoryEntry): MemoryEvidenceRef {
return {
memoryId: memory.id,
memoryKeyHash: memoryKey(memory),
identityKeyHash: workspaceMemoryIdentityKey(memory),
type: memory.type,
source: memory.source,
status: memory.status,
};
}
function pendingAppendedEvidence(memory: LongTermMemoryEntry): EvidenceEventInput {
return {
type: "pending_memory_appended",
phase: "pending_journal",
outcome: "accepted",
memory: memoryEvidenceRef(memory),
relations: [{ role: "pending", memory: memoryEvidenceRef(memory) }],
reasonCodes: ["pending_journal_append"],
textPreview: memory.text,
};
}
function pruneFrozenWorkspaceMemoryCache(now = Date.now()): void {
for (const [sessionID, cached] of frozenWorkspaceMemoryCache) {
if (now - cached.loadedAt > WORKSPACE_MEMORY_CACHE_LIMITS.frozenTtlMs) {
@@ -315,14 +253,7 @@ export const MemoryV2Plugin: Plugin = async (input) => {
if (!latestMessage?.id || processedForSession.has(latestMessage.id)) return;
const extraction = extractExplicitMemoriesWithEvidence(latestMessage.text);
await appendEvidenceEvents(directory, extraction.evidence.map(event => ({
...event,
sessionHash: sessionID,
messageHash: latestMessage.id,
})));
const memories = extraction.entries.map(memory => ({
const memories = extractExplicitMemories(latestMessage.text).map(memory => ({
...memory,
pendingOwnerSessionID: sessionID,
pendingMessageID: latestMessage.id,
@@ -335,11 +266,6 @@ export const MemoryV2Plugin: Plugin = async (input) => {
return state;
});
await appendPendingMemories(directory, memories);
await appendEvidenceEvents(directory, memories.map(memory => ({
...pendingAppendedEvidence(memory),
sessionHash: sessionID,
messageHash: latestMessage.id,
})));
}
if (decisions.length > 0) {
@@ -358,22 +284,6 @@ export const MemoryV2Plugin: Plugin = async (input) => {
rememberProcessedUserMessage(sessionID, latestMessage.id, processedForSession);
}
function dedupePendingPromotionMemories(
memories: LongTermMemoryEntry[],
): LongTermMemoryEntry[] {
const seen = new Set<string>();
const deduped: LongTermMemoryEntry[] = [];
for (const memory of memories) {
const key = memoryKey(memory);
if (seen.has(key)) continue;
seen.add(key);
deduped.push(memory);
}
return deduped;
}
async function promotePendingMemories(
sessionID?: string,
options: { includeUnownedJournal?: boolean; includeOwnedJournal?: boolean } = {},
@@ -391,45 +301,27 @@ export const MemoryV2Plugin: Plugin = async (input) => {
return false;
});
const pending = dedupePendingPromotionMemories([
const pending = [
...(sessionState?.pendingMemories ?? []),
...journalPending,
]);
];
if (pending.length === 0) return;
let beforeEntries: Awaited<ReturnType<typeof loadWorkspaceMemory>>["entries"] = [];
const updateResult = await updateWorkspaceMemoryWithAccounting(directory, workspaceMemory => {
beforeEntries = [...workspaceMemory.entries];
const existingByKey = new Map<string, { memory: typeof workspaceMemory.entries[number]; index: number }>();
workspaceMemory.entries.forEach((memory, index) => {
if (memory.status === "superseded") return;
existingByKey.set(memoryKey(memory), { memory, index });
});
const existingKeys = new Set(
workspaceMemory.entries
.filter(memory => memory.status !== "superseded")
.map(memory => memoryKey(memory)),
);
const promotedAt = Date.now();
for (const memory of pending) {
const key = memoryKey(memory);
const existing = existingByKey.get(key);
if (existing) {
const reinforced = reinforceMemory(
existing.memory,
sessionID ?? memory.pendingOwnerSessionID ?? "workspace-promotion",
promotedAt,
);
if (reinforced !== existing.memory) {
workspaceMemory.entries[existing.index] = reinforced;
existingByKey.set(key, { memory: reinforced, index: existing.index });
}
} else {
workspaceMemory.entries.push({
...memory,
retentionClock: memory.retentionClock ?? promotedAt,
});
existingByKey.set(key, {
memory: workspaceMemory.entries[workspaceMemory.entries.length - 1],
index: workspaceMemory.entries.length - 1,
});
if (!existingKeys.has(key)) {
workspaceMemory.entries.push(memory);
existingKeys.add(key);
}
}
@@ -453,20 +345,6 @@ export const MemoryV2Plugin: Plugin = async (input) => {
},
);
await appendEvidenceEvents(directory, [
...updateResult.evidence,
...promotionAccountingEvidenceEvents({
pending,
after: updateResult.store.entries,
events: updateResult.events,
accounting,
exhaustedRejectedKeys,
}),
].map(event => ({
...event,
sessionHash: sessionID,
})));
const sessionRemovalKeys = new Set([
...accounting.clearableKeys,
...exhaustedRejectedKeys,
@@ -541,12 +419,7 @@ export const MemoryV2Plugin: Plugin = async (input) => {
}
const store = await loadWorkspaceMemory(root);
const renderAccounting = accountWorkspaceMemoryRender(store);
const renderedPrompt = renderAccounting.prompt;
await appendEvidenceEvents(root, renderAccounting.evidence.map(event => ({
...event,
sessionHash: sessionID,
})));
const renderedPrompt = renderWorkspaceMemory(store);
frozenWorkspaceMemoryCache.set(sessionID, { store, renderedPrompt, loadedAt: now });
pruneFrozenWorkspaceMemoryCache(now);
return { store, renderedPrompt };
@@ -570,42 +443,38 @@ export const MemoryV2Plugin: Plugin = async (input) => {
const { sessionID } = hookInput;
if (!sessionID) return;
try {
pruneFrozenWorkspaceMemoryCache();
pruneProcessedUserMessagesCache();
pruneFrozenWorkspaceMemoryCache();
pruneProcessedUserMessagesCache();
// Sub-agents are short-lived - skip memory system
if (await isSubAgent(sessionID)) return;
// Sub-agents are short-lived - skip memory system
if (await isSubAgent(sessionID)) return;
// Process explicit user memory even on no-tool turns. Keep this after the
// sub-agent guard so child sessions never append to the parent journal.
await processLatestUserMessage(sessionID);
// Process explicit user memory even on no-tool turns. Keep this after the
// sub-agent guard so child sessions never append to the parent journal.
await processLatestUserMessage(sessionID);
// Before first snapshot in this session, promote durable unowned backlog from
// prior sessions. Current-turn owned explicit memory remains pending and only
// appears in hot state for this transform.
if (!frozenWorkspaceMemoryCache.has(sessionID) && await hasPendingJournalEntries(directory)) {
await promotePendingMemories(undefined, { includeUnownedJournal: true, includeOwnedJournal: false });
}
// Before first snapshot in this session, promote durable unowned backlog from
// prior sessions. Current-turn owned explicit memory remains pending and only
// appears in hot state for this transform.
if (!frozenWorkspaceMemoryCache.has(sessionID) && await hasPendingJournalEntries(directory)) {
await promotePendingMemories(undefined, { includeUnownedJournal: true, includeOwnedJournal: false });
}
// Get frozen workspace memory snapshot (loaded and rendered once per session)
const workspaceSnapshot = await getFrozenWorkspaceMemorySnapshot(directory, sessionID);
// Get frozen workspace memory snapshot (loaded and rendered once per session)
const workspaceSnapshot = await getFrozenWorkspaceMemorySnapshot(directory, sessionID);
// Get current hot session state
const sessionState = await loadSessionState(directory, sessionID);
// Get current hot session state
const sessionState = await loadSessionState(directory, sessionID);
// Inject frozen workspace memory snapshot
if (workspaceSnapshot.renderedPrompt) {
output.system.push(workspaceSnapshot.renderedPrompt);
}
// Inject frozen workspace memory snapshot
if (workspaceSnapshot.renderedPrompt) {
output.system.push(workspaceSnapshot.renderedPrompt);
}
// Render and inject hot session state
const hotPrompt = renderHotSessionState(sessionState, directory);
if (hotPrompt) {
output.system.push(hotPrompt);
}
} catch (error) {
await warnMemoryHook("chat.system.transform", error, directory);
// Render and inject hot session state
const hotPrompt = renderHotSessionState(sessionState, directory);
if (hotPrompt) {
output.system.push(hotPrompt);
}
},
@@ -618,54 +487,50 @@ export const MemoryV2Plugin: Plugin = async (input) => {
// Sub-agents don't need memory tracking
if (await isSubAgent(sessionID)) return;
try {
await updateSessionState(directory, sessionID, state => {
// Track active files from tool usage
if (toolName === "read" || toolName === "edit" || toolName === "write" || toolName === "grep") {
const files = extractActiveFiles(
toolName,
args as Record<string, unknown>,
toolOutput ?? ""
);
for (const { path, action } of files) {
touchActiveFile(state, path, action);
if (action === "edit" || action === "write") {
markErrorsMaybeFixedForFile(state, path, directory);
}
await updateSessionState(directory, sessionID, state => {
// Track active files from tool usage
if (toolName === "read" || toolName === "edit" || toolName === "write" || toolName === "grep") {
const files = extractActiveFiles(
toolName,
args as Record<string, unknown>,
toolOutput ?? ""
);
for (const { path, action } of files) {
touchActiveFile(state, path, action);
if (action === "edit" || action === "write") {
markErrorsMaybeFixedForFile(state, path, directory);
}
}
}
// Track errors from failed bash commands
if (toolName === "bash") {
const argsRecord = args as Record<string, unknown>;
const command: string = typeof argsRecord?.command === "string"
? argsRecord.command
: "";
const outputText: string = toolOutput ?? "";
// Track errors from failed bash commands
if (toolName === "bash") {
const argsRecord = args as Record<string, unknown>;
const command: string = typeof argsRecord?.command === "string"
? argsRecord.command
: "";
const outputText: string = toolOutput ?? "";
// Check if command succeeded - clear errors for that category
const exitCode = bashExitCode(hookOutput);
if (typeof exitCode !== "number") {
// Unknown exit status: do not extract and do not clear
} else if (exitCode === 0 && command) {
clearErrorsForSuccessfulCommand(state, command);
} else if (command) {
// Only extract errors for commands with explicit non-zero exit
const errors = extractErrorsFromBash(command, outputText);
for (const error of errors) {
upsertOpenError(state, error);
}
// Check if command succeeded - clear errors for that category
const exitCode = bashExitCode(hookOutput);
if (typeof exitCode !== "number") {
// Unknown exit status: do not extract and do not clear
} else if (exitCode === 0 && command) {
clearErrorsForSuccessfulCommand(state, command);
} else if (command) {
// Only extract errors for commands with explicit non-zero exit
const errors = extractErrorsFromBash(command, outputText);
for (const error of errors) {
upsertOpenError(state, error);
}
}
return state;
});
}
return state;
});
// Process explicit memory from latest user message
// Only process once per message ID
await processLatestUserMessage(sessionID);
} catch (error) {
await warnMemoryHook("tool.execute.after", error, directory);
}
// Process explicit memory from latest user message
// Only process once per message ID
await processLatestUserMessage(sessionID);
},
/**
@@ -683,110 +548,95 @@ export const MemoryV2Plugin: Plugin = async (input) => {
const { sessionID } = hookInput;
if (!sessionID) return;
try {
// Sub-agents don't need compaction support
if (await isSubAgent(sessionID)) return;
// Sub-agents don't need compaction support
if (await isSubAgent(sessionID)) return;
// Preserve context injected by other plugins that ran before us.
// Setting output.prompt bypasses the default prompt + context join,
// so we must explicitly carry forward any existing output.context.
const otherContext = output.context.filter(Boolean).join("\n\n");
// Preserve context injected by other plugins that ran before us.
// Setting output.prompt bypasses the default prompt + context join,
// so we must explicitly carry forward any existing output.context.
const otherContext = output.context.filter(Boolean).join("\n\n");
// Build our private context (workspace memory, hot state, todos)
const contextParts: string[] = [];
// Build our private context (workspace memory, hot state, todos)
const contextParts: string[] = [];
// 1. Frozen workspace memory snapshot
const workspaceSnapshot = await getFrozenWorkspaceMemorySnapshot(directory, sessionID);
if (workspaceSnapshot.renderedPrompt) {
contextParts.push(workspaceSnapshot.renderedPrompt);
}
// 2. Hot session state
const sessionState = await loadSessionState(directory, sessionID);
const hotPrompt = renderHotSessionState(sessionState, directory);
if (hotPrompt) {
contextParts.push(hotPrompt);
}
// 3. Pending todos from OpenCode
const todos = await pendingTodos(client, sessionID);
const todosPrompt = renderTodosForCompaction(todos);
if (todosPrompt) {
contextParts.push(todosPrompt);
}
// Combine: other plugins' context first, then our private context
const privateContext = [otherContext, ...contextParts]
.filter(Boolean)
.join("\n\n");
// Replace the default prompt entirely with our ---free template
output.prompt = buildCompactionPrompt(privateContext);
// Clear context array since we consumed it into output.prompt.
// Subsequent plugins that set output.prompt will also need to check
// output.context if they want to preserve other plugin contributions.
output.context.length = 0;
} catch (error) {
await warnMemoryHook("session.compacting", error, directory);
const workspaceSnapshot = await getFrozenWorkspaceMemorySnapshot(directory, sessionID);
if (workspaceSnapshot.renderedPrompt) {
contextParts.push(workspaceSnapshot.renderedPrompt);
}
// 2. Hot session state
const sessionState = await loadSessionState(directory, sessionID);
const hotPrompt = renderHotSessionState(sessionState, directory);
if (hotPrompt) {
contextParts.push(hotPrompt);
}
// 3. Pending todos from OpenCode
const todos = await pendingTodos(client, sessionID);
const todosPrompt = renderTodosForCompaction(todos);
if (todosPrompt) {
contextParts.push(todosPrompt);
}
// Combine: other plugins' context first, then our private context
const privateContext = [otherContext, ...contextParts]
.filter(Boolean)
.join("\n\n");
// Replace the default prompt entirely with our ---free template
output.prompt = buildCompactionPrompt(privateContext);
// Clear context array since we consumed it into output.prompt.
// Subsequent plugins that set output.prompt will also need to check
// output.context if they want to preserve other plugin contributions.
output.context.length = 0;
},
// Handle session events
event: async ({ event }) => {
if (event.type === "session.compacted") {
const sessionID = sessionIDFromEventProperties(event.properties);
if (!sessionID) return;
// Sub-agents don't need post-compaction processing
if (await isSubAgent(sessionID)) return;
// Parse latest compaction summary for memory candidates, stage them into
// durable pending journal, then promote pending memories.
const summary = await latestCompactionSummary(client, sessionID);
const candidates = summary ? parseWorkspaceMemoryCandidates(summary) : [];
if (candidates.length > 0) {
await appendPendingMemories(directory, candidates);
}
try {
const sessionID = sessionIDFromEventProperties(event.properties);
if (!sessionID) return;
// Sub-agents don't need post-compaction processing
if (await isSubAgent(sessionID)) return;
// Parse latest compaction summary for memory candidates, stage them into
// durable pending journal, then promote pending memories.
const summary = await latestCompactionSummary(client, sessionID);
const parseResult = summary
? parseWorkspaceMemoryCandidatesWithEvidence(summary, await workspaceIdentity(directory))
: { entries: [], evidence: [] };
await appendEvidenceEvents(directory, parseResult.evidence.map(event => ({
...event,
sessionHash: sessionID,
})));
const candidates = parseResult.entries;
if (candidates.length > 0) {
await appendPendingMemories(directory, candidates);
await appendEvidenceEvents(directory, candidates.map(memory => ({
...pendingAppendedEvidence(memory),
sessionHash: sessionID,
})));
}
await promotePendingMemories(sessionID, { includeUnownedJournal: true });
} catch (error) {
} catch {
// Keep pending memories in session/journal for retry on next event/session.
await warnMemoryHook("event.session.compacted", error, directory);
}
}
if (event.type === "session.deleted") {
try {
const sessionID = sessionIDFromEventProperties(event.properties);
if (sessionID) {
const sessionID = sessionIDFromEventProperties(event.properties);
if (sessionID) {
// Promote pending memories before deleting per-session state.
// If promotion fails, leave session state and journal intact.
let promoted = false;
let promoted = false;
try {
await promotePendingMemories(sessionID, { includeOwnedJournal: true, includeUnownedJournal: false });
promoted = true;
} catch {
return;
} finally {
if (promoted) {
frozenWorkspaceMemoryCache.delete(sessionID);
processedUserMessages.delete(sessionID);
sessionParentCache.delete(sessionID);
}
await rm(await sessionStatePath(directory, sessionID), { force: true });
}
} catch (error) {
await warnMemoryHook("event.session.deleted", error, directory);
await rm(await sessionStatePath(directory, sessionID), { force: true });
}
}
},
+5 -118
View File
@@ -2,7 +2,6 @@ import type { LongTermMemoryEntry } from "./types.ts";
import { memoryKey } from "./pending-journal.ts";
import type { MemoryConsolidationEvent } from "./workspace-memory.ts";
import { workspaceMemoryIdentityKey } from "./workspace-memory.ts";
import type { EvidenceEventInput, MemoryEvidenceRef } from "./evidence-log.ts";
export type PendingPromotionAccounting = {
promotedKeys: Set<string>;
@@ -60,7 +59,7 @@ export function accountPendingPromotions(input: {
continue;
}
if (terminal.reason === "rejected_capacity") {
if (terminal.reason === "rejected_capacity" || terminal.reason === "rejected_stale") {
rejectedKeys.add(key);
continue;
}
@@ -81,7 +80,10 @@ export function accountPendingPromotions(input: {
...input.pending
.filter(memory => {
const terminal = terminalEventByKey.get(memoryKey(memory));
return memory.source === "compaction" && terminal?.reason === "rejected_capacity";
return memory.source === "compaction" && (
terminal?.reason === "rejected_capacity" ||
terminal?.reason === "rejected_stale"
);
})
.map(memory => memoryKey(memory)),
]);
@@ -106,118 +108,3 @@ export function accountPendingPromotions(input: {
clearableKeys,
};
}
function memoryRef(memory: LongTermMemoryEntry | undefined): MemoryEvidenceRef | undefined {
if (!memory) return undefined;
return {
memoryId: memory.id,
memoryKeyHash: memoryKey(memory),
identityKeyHash: workspaceMemoryIdentityKey(memory),
type: memory.type,
source: memory.source,
status: memory.status,
};
}
function retainedMemoryFor(
pending: LongTermMemoryEntry,
event: MemoryConsolidationEvent | undefined,
after: LongTermMemoryEntry[],
): LongTermMemoryEntry | undefined {
if (event?.retainedId) {
const byId = after.find(memory => memory.id === event.retainedId);
if (byId) return byId;
}
const exactKey = memoryKey(pending);
const identityKey = workspaceMemoryIdentityKey(pending);
return after.find(memory => memory.status !== "superseded" && (
memoryKey(memory) === exactKey || workspaceMemoryIdentityKey(memory) === identityKey
));
}
function promotionEventBase(
type: EvidenceEventInput["type"],
outcome: EvidenceEventInput["outcome"],
memory: LongTermMemoryEntry,
reasonCodes: string[],
): EvidenceEventInput {
return {
type,
phase: "promotion",
outcome,
memory: memoryRef(memory),
reasonCodes,
textPreview: memory.text,
};
}
export function promotionAccountingEvidenceEvents(input: {
pending: LongTermMemoryEntry[];
after: LongTermMemoryEntry[];
events?: MemoryConsolidationEvent[];
accounting: PendingPromotionAccounting;
exhaustedRejectedKeys?: Set<string>;
}): EvidenceEventInput[] {
const terminalByKey = new Map((input.events ?? []).map(event => [event.memoryKey, event]));
const exhaustedRejectedKeys = input.exhaustedRejectedKeys ?? new Set<string>();
const evidence: EvidenceEventInput[] = [];
for (const pending of input.pending) {
const key = memoryKey(pending);
const terminal = terminalByKey.get(key);
const retained = retainedMemoryFor(pending, terminal, input.after);
if (input.accounting.promotedKeys.has(key)) {
evidence.push({
...promotionEventBase("promotion_promoted", "promoted", pending, ["new_workspace_entry"]),
relations: [
{ role: "promoted", memory: memoryRef(retained ?? pending) },
],
});
continue;
}
if (input.accounting.absorbedKeys.has(key)) {
const exact = terminal?.reason !== "absorbed_identity";
evidence.push({
...promotionEventBase(
exact ? "promotion_absorbed_exact" : "promotion_absorbed_identity",
"absorbed",
pending,
[exact ? "same_exact_key" : "same_identity_key"],
),
relations: [
{ role: "absorbed" as const, memory: memoryRef(pending) },
{ role: "retained" as const, memory: memoryRef(retained) },
].filter(relation => relation.memory),
});
continue;
}
if (input.accounting.supersededKeys.has(key)) {
evidence.push({
...promotionEventBase("promotion_superseded", "superseded", pending, ["superseded_existing"]),
relations: [
{ role: "superseded" as const, memory: memoryRef(pending) },
{ role: "superseded_by" as const, memory: memoryRef(retained) },
].filter(relation => relation.memory),
});
continue;
}
if (input.accounting.rejectedKeys.has(key)) {
evidence.push(promotionEventBase("promotion_rejected_capacity", "rejected", pending, ["capacity_rejected"]));
if (input.accounting.retryableRejectedKeys.has(key)) {
evidence.push(promotionEventBase(
exhaustedRejectedKeys.has(key) ? "promotion_retry_exhausted" : "promotion_retry_scheduled",
exhaustedRejectedKeys.has(key) ? "exhausted" : "retried",
pending,
[exhaustedRejectedKeys.has(key) ? "max_attempts_reached" : "retryable_capacity_rejection"],
));
}
}
}
return evidence;
}
-148
View File
@@ -1,148 +0,0 @@
import type { LongTermMemoryEntry, WorkspaceMemoryStore } from "./types.ts";
// Retention decay model constants (v1.5)
export const BASE_HALF_LIFE_DAYS = 45;
export const REINFORCEMENT_HALFLIFE_FACTOR = 0.85;
export const REINFORCEMENT_MAX_COUNT = 6;
export const REINFORCEMENT_MIN_INTERVAL_MS = 60 * 60 * 1000; // 1 hour
export const WORKSPACE_DORMANT_AFTER_DAYS = 14;
export const DORMANT_DECAY_MULTIPLIER = 0.25;
export const DAY_MS = 24 * 60 * 60 * 1000;
export const TYPE_FACTOR = {
reference: 1.0,
project: 1.25,
feedback: 2.25,
decision: 2.5,
} as const;
export const SOURCE_FACTOR = {
compaction: 1.0,
manual: 1.4,
explicit: 2.0,
} as const;
export const USER_IMPORTANCE_FACTOR = {
low: 0.7,
normal: 1.0,
high: 1.5,
} as const;
export const RETENTION_TYPE_MAX = {
feedback: 10,
decision: 10,
project: 8,
reference: 6,
} as const;
export function calculateInitialStrength(memory: LongTermMemoryEntry): number {
const typeFactor = TYPE_FACTOR[memory.type] ?? 1.0;
const sourceFactor = SOURCE_FACTOR[memory.source] ?? 1.0;
const importanceFactor = USER_IMPORTANCE_FACTOR[memory.userImportance ?? "normal"] ?? 1.0;
return typeFactor * sourceFactor * importanceFactor;
}
export function calculateEffectiveHalfLife(memory: LongTermMemoryEntry): number {
const reinforcementCount = Math.min(
memory.reinforcementCount ?? 0,
REINFORCEMENT_MAX_COUNT,
);
const factor = Math.pow(REINFORCEMENT_HALFLIFE_FACTOR, reinforcementCount);
return BASE_HALF_LIFE_DAYS / factor;
}
function timestampMs(value: unknown, fallback: number): number {
const ms = typeof value === "number" ? value : new Date(String(value)).getTime();
return Number.isFinite(ms) ? ms : fallback;
}
export function calculateRetentionStrength(
memory: LongTermMemoryEntry,
now: number,
lastActivityAt?: string,
): number {
const initialStrength = calculateInitialStrength(memory);
const effectiveHalfLife = calculateEffectiveHalfLife(memory);
// Use retentionClock if available, fallback to updatedAt.
const retentionStart = Number.isFinite(memory.retentionClock)
? memory.retentionClock
: memory.updatedAt ?? memory.createdAt;
const createdAtMs = timestampMs(retentionStart, now);
const effectiveAgeDays = calculateEffectiveAgeDays(createdAtMs, now, lastActivityAt);
// Calculate strength using exponential decay.
const strength = initialStrength * Math.pow(2, -effectiveAgeDays / effectiveHalfLife);
return Number.isFinite(strength) ? Math.max(0, strength) : 0;
}
export function calculateDormantDays(store: WorkspaceMemoryStore, now: number): number {
const lastActivity = store.lastActivityAt
? new Date(store.lastActivityAt).getTime()
: now;
if (!Number.isFinite(lastActivity)) return 0;
const daysSinceActivity = (now - lastActivity) / DAY_MS;
return Math.max(0, daysSinceActivity);
}
export function calculateEffectiveAgeDays(
entryStartMs: number,
now: number,
lastActivityAt?: string,
): number {
const wallAgeDays = Math.max(0, (now - entryStartMs) / DAY_MS);
if (!lastActivityAt) return wallAgeDays;
const lastActivityMs = new Date(lastActivityAt).getTime();
if (!Number.isFinite(lastActivityMs)) return wallAgeDays;
const dormantStartMs = lastActivityMs + WORKSPACE_DORMANT_AFTER_DAYS * DAY_MS;
const overlapStartMs = Math.max(entryStartMs, dormantStartMs);
const dormantOverlapDays = Math.max(0, (now - overlapStartMs) / DAY_MS);
const activeDays = wallAgeDays - dormantOverlapDays;
return activeDays + dormantOverlapDays * DORMANT_DECAY_MULTIPLIER;
}
function isSameUTCCalendarDay(ts1: number, ts2: number): boolean {
const d1 = new Date(ts1);
const d2 = new Date(ts2);
return d1.getUTCFullYear() === d2.getUTCFullYear()
&& d1.getUTCMonth() === d2.getUTCMonth()
&& d1.getUTCDate() === d2.getUTCDate();
}
export function reinforceMemory(
memory: LongTermMemoryEntry,
sessionId: string,
now: number,
): LongTermMemoryEntry {
if (memory.lastReinforcedSessionID === sessionId) {
return memory;
}
// Calendar-day diversity gate (OQ-2): same UTC day = no reinforcement.
if (memory.lastReinforcedAt && isSameUTCCalendarDay(memory.lastReinforcedAt, now)) {
return memory;
}
if (memory.lastReinforcedAt && now - memory.lastReinforcedAt < REINFORCEMENT_MIN_INTERVAL_MS) {
return memory;
}
if ((memory.reinforcementCount ?? 0) >= REINFORCEMENT_MAX_COUNT) {
return memory;
}
return {
...memory,
reinforcementCount: (memory.reinforcementCount ?? 0) + 1,
lastReinforcedAt: now,
lastReinforcedSessionID: sessionId,
retentionClock: now,
};
}
+1 -70
View File
@@ -3,66 +3,17 @@ import { randomUUID } from "crypto";
import { mkdir, open, readFile, rename, rm, stat, writeFile } from "fs/promises";
import type { FileHandle } from "fs/promises";
import { dirname } from "path";
import { appendEvidenceEventForWorkspaceKey } from "./evidence-log.ts";
const fileLocks = new Map<string, Promise<unknown>>();
const LOCK_WAIT_TIMEOUT_MS = 5000;
const LOCK_STALE_MS = 30_000;
const LOCK_HEARTBEAT_MS = 1_000;
function workspaceKeyFromStorePath(path: string): string | undefined {
return path.match(/[\\/]opencode-working-memory[\\/]workspaces[\\/]([a-f0-9]{16})[\\/]/i)?.[1];
}
function storeKindFromPath(path: string): string {
if (path.endsWith("workspace-memory.json")) return "workspace_memory";
if (path.endsWith("workspace-pending-journal.json")) return "pending_journal";
if (path.includes(`${"/"}sessions${"/"}`) || path.includes(`${"\\"}sessions${"\\"}`)) return "session_state";
return "unknown";
}
async function emitStorageEvidence(path: string, event: Parameters<typeof appendEvidenceEventForWorkspaceKey>[1]): Promise<void> {
const key = workspaceKeyFromStorePath(path);
if (!key) return;
await appendEvidenceEventForWorkspaceKey(key, event).catch(() => undefined);
}
async function quarantineCorruptJSON(path: string): Promise<string | null> {
const quarantinePath = `${path}.corrupt-${Date.now()}-${process.pid}-${randomUUID()}`;
try {
await rename(path, quarantinePath);
return quarantinePath;
} catch {
return null;
}
}
export async function readJSON<T>(path: string, fallback: () => T): Promise<T> {
if (!existsSync(path)) return fallback();
try {
return JSON.parse(await readFile(path, "utf8")) as T;
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
const quarantinePath = await quarantineCorruptJSON(path);
if (quarantinePath) {
console.error(`[memory] invalid JSON in ${path}; quarantined to ${quarantinePath}: ${message}`);
await emitStorageEvidence(path, {
type: "storage_corrupt_json_quarantined",
phase: "storage",
outcome: "quarantined",
reasonCodes: ["invalid_json"],
details: {
storeKind: storeKindFromPath(path),
quarantined: true,
},
});
} else {
console.error(`[memory] invalid JSON in ${path}; using fallback without quarantine: ${message}`);
}
} catch {
return fallback();
}
}
@@ -131,30 +82,10 @@ async function withFileLock<T>(path: string, fn: () => Promise<T>): Promise<T> {
if (await isLockStale(lockPath)) {
await rm(lockPath, { force: true });
await emitStorageEvidence(path, {
type: "storage_stale_lock_recovered",
phase: "storage",
outcome: "recovered",
reasonCodes: ["stale_lock"],
details: {
storeKind: storeKindFromPath(path),
waitMs: Date.now() - started,
},
});
continue;
}
if (Date.now() - started > LOCK_WAIT_TIMEOUT_MS) {
await emitStorageEvidence(path, {
type: "storage_lock_timeout",
phase: "storage",
outcome: "failed",
reasonCodes: ["lock_wait_timeout"],
details: {
storeKind: storeKindFromPath(path),
waitMs: LOCK_WAIT_TIMEOUT_MS,
},
});
throw new Error(`Timed out waiting for lock ${lockPath}`);
}
+3 -10
View File
@@ -20,12 +20,6 @@ export type LongTermMemoryEntry = {
promotionAttempts?: number;
lastPromotionAttemptAt?: string;
lastPromotionFailureReason?: string;
retentionClock?: number; // Unix timestamp when retention started
reinforcementCount?: number; // Number of times this memory was reinforced
lastReinforcedAt?: number; // Unix timestamp of last reinforcement
lastReinforcedSessionID?: string;
userImportance?: "low" | "normal" | "high";
safetyCritical?: boolean;
};
export type WorkspaceMemoryStore = {
@@ -41,7 +35,6 @@ export type WorkspaceMemoryStore = {
entries: LongTermMemoryEntry[];
migrations?: string[];
updatedAt: string;
lastActivityAt?: string;
};
export type PendingMemoryJournalStore = {
@@ -95,15 +88,15 @@ export type SessionState = {
};
export const LONG_TERM_LIMITS = {
maxRenderedChars: 3600,
targetRenderedChars: 3000,
maxRenderedChars: 5200,
targetRenderedChars: 4200,
maxEntries: 28,
maxEntryTextChars: 260,
maxRationaleChars: 180,
} as const;
export const HOT_STATE_LIMITS = {
maxRenderedChars: 700,
maxRenderedChars: 1200,
maxActiveFilesStored: 20,
maxActiveFilesRendered: 8,
maxOpenErrorsStored: 5,
+69 -336
View File
@@ -6,26 +6,19 @@ import { migrationLogPath, workspaceKey, workspaceMemoryPath } from "./paths.ts"
import { atomicWriteJSON, readJSON, updateJSON } from "./storage.ts";
import { assessMemoryQuality, isHardQualityReason, isProgressSnapshotViolation } from "./memory-quality.ts";
import { redactCredentials } from "./redaction.ts";
import {
RETENTION_TYPE_MAX,
calculateRetentionStrength,
reinforceMemory,
} from "./retention.ts";
import type { EvidenceEventInput, MemoryEvidenceRef } from "./evidence-log.ts";
import { appendEvidenceEvents } from "./evidence-log.ts";
// Minimum length for workspace_memory envelope: <workspace_memory>\n...\n</workspace_memory>
const MIN_ENVELOPE_LENGTH = 80;
const MIGRATION_ID = "2026-04-26-p0-cleanup";
const QUALITY_CLEANUP_MIGRATION_ID = "2026-04-28-quality-cleanup";
const RETENTION_CLOCK_BACKFILL_MIGRATION_ID = "2026-05-01-retention-clock-backfill";
export type MemoryConsolidationReason =
| "promoted"
| "absorbed_exact"
| "absorbed_identity"
| "superseded_existing"
| "rejected_capacity";
| "rejected_capacity"
| "rejected_stale";
export type MemoryConsolidationEvent = {
memoryKey: string;
@@ -41,7 +34,6 @@ export type LongTermLimitResult = {
dropped: MemoryConsolidationEvent[];
absorbed: MemoryConsolidationEvent[];
superseded: MemoryConsolidationEvent[];
evidence: EvidenceEventInput[];
};
export type WorkspaceMemoryNormalizationResult = LongTermLimitResult & {
@@ -49,16 +41,6 @@ export type WorkspaceMemoryNormalizationResult = LongTermLimitResult & {
events: MemoryConsolidationEvent[];
};
export type WorkspaceMemoryRenderAccounting = {
rendered: LongTermMemoryEntry[];
omitted: Array<{
memory: LongTermMemoryEntry;
reason: "superseded" | "type_cap" | "global_cap" | "char_budget" | "empty_render_budget";
}>;
evidence: EvidenceEventInput[];
prompt: string;
};
export type QualityCleanupMigrationLogEntry = {
migrationId: string;
timestamp: string;
@@ -74,19 +56,7 @@ export type QualityCleanupMigrationLogEntry = {
afterStatus: "superseded";
};
export type P0CleanupMigrationResult = {
store: WorkspaceMemoryStore;
events: EvidenceEventInput[];
};
export type QualityCleanupMigrationResult = {
store: WorkspaceMemoryStore;
events: QualityCleanupMigrationLogEntry[];
evidence: EvidenceEventInput[];
};
export async function emptyWorkspaceMemory(root: string): Promise<WorkspaceMemoryStore> {
const nowIso = new Date().toISOString();
return {
version: 1,
workspace: { root, key: await workspaceKey(root) },
@@ -96,8 +66,7 @@ export async function emptyWorkspaceMemory(root: string): Promise<WorkspaceMemor
},
entries: [],
migrations: [],
updatedAt: nowIso,
lastActivityAt: nowIso,
updatedAt: new Date().toISOString(),
};
}
@@ -116,7 +85,6 @@ export async function loadWorkspaceMemory(root: string): Promise<WorkspaceMemory
entries: Array.isArray(loaded.entries) ? loaded.entries : [],
migrations: Array.isArray(loaded.migrations) ? loaded.migrations : [],
updatedAt: loaded.updatedAt ?? fallback.updatedAt,
lastActivityAt: loaded.lastActivityAt ?? loaded.updatedAt ?? fallback.lastActivityAt,
};
// Always normalize on load so redaction/migrations are always-on.
@@ -126,14 +94,6 @@ export async function loadWorkspaceMemory(root: string): Promise<WorkspaceMemory
// writes for ordering/capacity/timestamp-only normalization.
if (hasSecurityOrMigrationChange(store, normalized.store)) {
await atomicWriteJSON(path, normalized.store);
// loadWorkspaceMemory has a narrow migration side effect: when a first-load
// migration actually supersedes entries, append evidence for those storage
// changes. Migration IDs keep this idempotent on repeated loads.
const migrationEvidence = normalized.evidence.filter(event => event.type === "memory_migration_superseded");
if (migrationEvidence.length > 0) {
await appendEvidenceEvents(root, migrationEvidence);
}
}
return normalized.store;
@@ -150,12 +110,10 @@ function hasSecurityOrMigrationChange(
if (beforeEntry.text !== afterEntry.text) return true;
if ((beforeEntry.rationale ?? "") !== (afterEntry.rationale ?? "")) return true;
if (beforeEntry.status !== afterEntry.status) return true;
if ((beforeEntry.retentionClock ?? null) !== (afterEntry.retentionClock ?? null)) return true;
}
const beforeMigrations = JSON.stringify(before.migrations ?? []);
const afterMigrations = JSON.stringify(after.migrations ?? []);
if ((before.lastActivityAt ?? "") !== (after.lastActivityAt ?? "")) return true;
return beforeMigrations !== afterMigrations;
}
@@ -179,15 +137,8 @@ export async function updateWorkspaceMemoryWithAccounting(
const fallback = await emptyWorkspaceMemory(root);
let finalResult: WorkspaceMemoryNormalizationResult | undefined;
const store = await updateJSON(path, () => fallback, async current => {
const currentNormalization = await normalizeWorkspaceMemoryWithAccounting(root, current);
const currentMigrationEvidence = currentNormalization.evidence.filter(event => event.type === "memory_migration_superseded");
finalResult = await normalizeWorkspaceMemoryWithAccounting(root, await updater(currentNormalization.store));
if (currentMigrationEvidence.length > 0) {
finalResult = {
...finalResult,
evidence: [...currentMigrationEvidence, ...finalResult.evidence],
};
}
const normalized = await normalizeWorkspaceMemory(root, current);
finalResult = await normalizeWorkspaceMemoryWithAccounting(root, await updater(normalized));
return finalResult.store;
});
@@ -197,7 +148,6 @@ export async function updateWorkspaceMemoryWithAccounting(
dropped: [],
absorbed: [],
superseded: [],
evidence: [],
events: [],
};
}
@@ -213,8 +163,7 @@ export async function normalizeWorkspaceMemoryWithAccounting(
root: string,
store: WorkspaceMemoryStore,
): Promise<WorkspaceMemoryNormalizationResult> {
const nowMs = Date.now();
const nowIso = new Date(nowMs).toISOString();
const nowIso = new Date().toISOString();
let result: WorkspaceMemoryStore = {
...store,
@@ -251,12 +200,10 @@ export async function normalizeWorkspaceMemoryWithAccounting(
const beforeQualityCleanup = result;
const qualityCleanup = runMigrationQualityCleanup(result, nowIso);
result = qualityCleanup.store;
let migrationEvidence: EvidenceEventInput[] = [];
let skipRemainingMigrations = false;
if (qualityCleanup.events.length > 0) {
try {
await appendQualityCleanupMigrationLog(qualityCleanup.events);
migrationEvidence = [...migrationEvidence, ...qualityCleanup.evidence];
} catch (error) {
console.error("[memory] failed to write quality cleanup migration log:", error);
console.error("[memory] aborting migration to maintain audit trail integrity");
@@ -265,18 +212,7 @@ export async function normalizeWorkspaceMemoryWithAccounting(
}
}
if (!skipRemainingMigrations) {
const p0Cleanup = runMigrationP0Cleanup(result, nowIso);
result = p0Cleanup.store;
migrationEvidence = [...migrationEvidence, ...p0Cleanup.events];
}
result.entries = result.entries.map(entry => backfillRetentionClock(entry, nowMs));
if (!result.migrations.includes(RETENTION_CLOCK_BACKFILL_MIGRATION_ID)) {
result = {
...result,
migrations: [...result.migrations, RETENTION_CLOCK_BACKFILL_MIGRATION_ID],
updatedAt: nowIso,
};
result = runMigrationP0Cleanup(result, nowIso);
}
// P0 accounting only considers active entries. Entries that were already
@@ -285,13 +221,12 @@ export async function normalizeWorkspaceMemoryWithAccounting(
// archived as superseded records in this wave.
const activeEntries = result.entries.filter(entry => entry.status !== "superseded");
const supersededEntries = result.entries.filter(entry => entry.status === "superseded");
const accounting = enforceLongTermLimitsWithAccounting(activeEntries, result);
const accounting = enforceLongTermLimitsWithAccounting(activeEntries);
const normalizedStore = {
...result,
entries: [...accounting.kept, ...supersededEntries],
updatedAt: nowIso,
lastActivityAt: nowIso,
};
return {
@@ -300,64 +235,38 @@ export async function normalizeWorkspaceMemoryWithAccounting(
dropped: accounting.dropped,
absorbed: accounting.absorbed,
superseded: accounting.superseded,
evidence: [...migrationEvidence, ...accounting.evidence],
events: [...accounting.dropped, ...accounting.absorbed, ...accounting.superseded],
};
}
function backfillRetentionClock(entry: LongTermMemoryEntry, nowMs: number): LongTermMemoryEntry {
if (Number.isFinite(entry.retentionClock)) {
return entry;
}
const createdAtMs = new Date(entry.createdAt).getTime();
if (Number.isFinite(createdAtMs)) {
return { ...entry, retentionClock: createdAtMs };
}
const updatedAtMs = new Date(entry.updatedAt).getTime();
if (Number.isFinite(updatedAtMs)) {
return { ...entry, retentionClock: updatedAtMs };
}
return { ...entry, retentionClock: nowMs };
}
export function runMigrationP0Cleanup(
store: WorkspaceMemoryStore,
nowIso: string,
): P0CleanupMigrationResult {
): WorkspaceMemoryStore {
if (store.migrations?.includes(MIGRATION_ID)) {
return { store, events: [] };
return store;
}
const events: EvidenceEventInput[] = [];
const entries = store.entries.map(entry => {
if (entry.source !== "compaction") return entry;
if (entry.type !== "project") return entry;
if (entry.status === "superseded") return entry;
if (isProgressSnapshotViolation(entry.text)) {
const superseded = {
return {
...entry,
status: "superseded" as const,
updatedAt: nowIso,
};
events.push(migrationSupersededEvidence(superseded, ["migration:p0_cleanup"], MIGRATION_ID));
return superseded;
}
return entry;
});
return {
store: {
...store,
entries,
migrations: [...(store.migrations || []), MIGRATION_ID],
updatedAt: nowIso,
},
events,
...store,
entries,
migrations: [...(store.migrations || []), MIGRATION_ID],
updatedAt: nowIso,
};
}
@@ -371,13 +280,12 @@ async function appendQualityCleanupMigrationLog(events: QualityCleanupMigrationL
export function runMigrationQualityCleanup(
store: WorkspaceMemoryStore,
nowIso: string,
): QualityCleanupMigrationResult {
): { store: WorkspaceMemoryStore; events: QualityCleanupMigrationLogEntry[] } {
if (store.migrations?.includes(QUALITY_CLEANUP_MIGRATION_ID)) {
return { store, events: [], evidence: [] };
return { store, events: [] };
}
const events: QualityCleanupMigrationLogEntry[] = [];
const evidence: EvidenceEventInput[] = [];
let changed = false;
const entries = store.entries.map(entry => {
if (entry.source !== "compaction") return entry;
@@ -411,19 +319,12 @@ export function runMigrationQualityCleanup(
...hardReasons.map(reason => `quality:${reason}`),
]);
const superseded = {
return {
...entry,
status: "superseded" as const,
updatedAt: nowIso,
tags: [...tags],
};
evidence.push(migrationSupersededEvidence(
superseded,
["migration:quality_cleanup", ...hardReasons.map(reason => `quality:${reason}`)],
QUALITY_CLEANUP_MIGRATION_ID,
{ hardReasons },
));
return superseded;
});
return {
@@ -434,7 +335,6 @@ export function runMigrationQualityCleanup(
updatedAt: changed ? nowIso : store.updatedAt,
},
events,
evidence,
};
}
@@ -547,52 +447,17 @@ function consolidationEvent(
};
}
function capacityRemovalEvidence(
memory: LongTermMemoryEntry,
reason: "type_cap" | "global_cap" | "capacity",
): EvidenceEventInput {
return {
type: "memory_removed_capacity",
phase: "storage",
outcome: "removed",
reasonCodes: [reason],
memory: memoryEvidenceRef(memory),
relations: [{
role: "removed",
memory: memoryEvidenceRef(memory),
}],
details: {
type: memory.type,
globalCap: LONG_TERM_LIMITS.maxEntries,
...(reason === "type_cap" ? { typeCap: RETENTION_TYPE_MAX[memory.type] } : {}),
...(typeof memory.retentionClock === "number" && Number.isFinite(memory.retentionClock) ? { retentionClock: memory.retentionClock } : {}),
...(memory.createdAt ? { createdAt: memory.createdAt } : {}),
...(memory.source ? { source: memory.source } : {}),
},
};
}
/** Check if entry should be pruned by age (for compaction/manual entries only) */
function isPrunableByAge(entry: LongTermMemoryEntry, now: number): boolean {
// Never prune feedback or explicit entries
if (entry.type === "feedback") return false;
if (entry.source === "explicit") return false;
if (!entry.staleAfterDays) return false;
function migrationSupersededEvidence(
memory: LongTermMemoryEntry,
reasonCodes: string[],
migrationId: string,
details: EvidenceEventInput["details"] = {},
): EvidenceEventInput {
return {
type: "memory_migration_superseded",
phase: "storage",
outcome: "superseded",
memory: memoryEvidenceRef(memory),
relations: [{ role: "superseded", memory: memoryEvidenceRef(memory) }],
reasonCodes,
details: {
migrationId,
type: memory.type,
source: memory.source,
...details,
},
textPreview: memory.text,
};
const createdAt = new Date(entry.createdAt).getTime();
const ageDays = (now - createdAt) / 86400000;
const grace = 30; // 30-day grace period
return ageDays > entry.staleAfterDays + grace;
}
/** Choose better memory when identity/topic keys conflict */
@@ -630,82 +495,42 @@ export function enforceLongTermLimits(entries: LongTermMemoryEntry[]): LongTermM
return enforceLongTermLimitsWithAccounting(entries).kept;
}
export function enforceLongTermLimitsWithAccounting(
entries: LongTermMemoryEntry[],
store?: WorkspaceMemoryStore,
): LongTermLimitResult {
export function enforceLongTermLimitsWithAccounting(entries: LongTermMemoryEntry[]): LongTermLimitResult {
const now = Date.now();
const lastActivityAt = store?.lastActivityAt;
const staleDropped: MemoryConsolidationEvent[] = [];
// Phase 1: filter active entries and trim text. Retention removal is by
// strength/cap competition, not hard stale pruning.
// Phase 1: filter active, prune by age
const phase1: LongTermMemoryEntry[] = [];
for (const entry of entries) {
if (entry.status === "superseded") continue;
if (isPrunableByAge(entry, now)) {
staleDropped.push(consolidationEvent(entry, "rejected_stale"));
continue;
}
phase1.push({ ...entry, text: entry.text.slice(0, LONG_TERM_LIMITS.maxEntryTextChars) });
}
const dedupeResult = dedupeLongTermEntriesWithAccounting(phase1);
const sorted = [...dedupeResult.kept].sort((a, b) => compareLongTermMemoryForRetention(a, b, now, lastActivityAt));
const capped = applyTypeMaxCaps(sorted);
const kept = capped.slice(0, LONG_TERM_LIMITS.maxEntries);
const sorted = [...dedupeResult.kept].sort(compareLongTermMemoryForRetention);
const kept = sorted.slice(0, LONG_TERM_LIMITS.maxEntries);
const keptIds = new Set(kept.map(entry => entry.id));
const cappedIds = new Set(capped.map(entry => entry.id));
const typeCapLosers = sorted.filter(entry => !cappedIds.has(entry.id));
const globalCapLosers = capped.filter(entry => !keptIds.has(entry.id));
const capacityEvidence: EvidenceEventInput[] = [
...typeCapLosers.map(entry => capacityRemovalEvidence(entry, "type_cap")),
...globalCapLosers.map(entry => capacityRemovalEvidence(entry, "global_cap")),
];
const capacityDropped = sorted
.filter(entry => !keptIds.has(entry.id))
.map(entry => consolidationEvent(entry, "rejected_capacity"));
return {
kept,
dropped: [...dedupeResult.dropped, ...capacityDropped],
dropped: [...staleDropped, ...dedupeResult.dropped, ...capacityDropped],
absorbed: dedupeResult.absorbed,
superseded: dedupeResult.superseded,
evidence: [...dedupeResult.evidence, ...capacityEvidence],
};
}
function applyTypeMaxCaps(entries: LongTermMemoryEntry[]): LongTermMemoryEntry[] {
return applyTypeMaxCapsWithOmissions(entries).kept;
}
function applyTypeMaxCapsWithOmissions(entries: LongTermMemoryEntry[]): { kept: LongTermMemoryEntry[]; omitted: LongTermMemoryEntry[] } {
const capped: LongTermMemoryEntry[] = [];
const omitted: LongTermMemoryEntry[] = [];
const typeCounts: Partial<Record<LongTermMemoryEntry["type"], number>> = {};
for (const entry of entries) {
const count = typeCounts[entry.type] ?? 0;
const max = RETENTION_TYPE_MAX[entry.type] ?? Infinity;
if (count >= max) {
omitted.push(entry);
continue;
}
capped.push(entry);
typeCounts[entry.type] = count + 1;
}
return { kept: capped, omitted };
}
export function dedupeLongTermEntriesWithAccounting(entries: LongTermMemoryEntry[]): LongTermLimitResult {
const now = Date.now();
const absorbed: MemoryConsolidationEvent[] = [];
const superseded: MemoryConsolidationEvent[] = [];
const evidence: EvidenceEventInput[] = [];
// For project/reference/feedback: dedupe by concrete identity or exact canonical text.
// Feedback is grouped with project/reference for entity dedupe, but
// workspaceMemoryIdentityKey() returns exact key for feedback (no concrete
// identity extraction). This means feedback absorbed_identity is currently
// impossible. When identity key extraction is extended to all types, feedback
// with matching concrete identifiers will correctly produce absorbed_identity.
const projectRefEntries = entries.filter(e => e.type === "project" || e.type === "reference" || e.type === "feedback");
// Build identity key dedup for project/reference/feedback.
@@ -722,16 +547,12 @@ export function dedupeLongTermEntriesWithAccounting(entries: LongTermMemoryEntry
const reason = workspaceMemoryExactKey(entry) === workspaceMemoryExactKey(existing)
? "absorbed_exact" as const
: "absorbed_identity" as const;
const reinforced = reinforceMemory(
retained,
reinforcementSessionId(retained, dropped),
now,
);
const reinforcedEvent = reinforcementEvidence(retained, dropped, reinforced, reason);
if (reinforcedEvent) evidence.push(reinforcedEvent);
absorbed.push(consolidationEvent(dropped, reason, reinforced));
entityDeduped.set(key, reinforced);
absorbed.push(consolidationEvent(dropped, reason, retained));
if (retained === entry) {
entityDeduped.set(key, entry);
}
}
}
@@ -749,22 +570,17 @@ export function dedupeLongTermEntriesWithAccounting(entries: LongTermMemoryEntry
const dropped = retained === entry ? existing : entry;
const reason = workspaceMemoryExactKey(entry) === workspaceMemoryExactKey(existing)
? "absorbed_exact" as const
: "superseded_existing" as const; // v1.5.4 placeholder: unreachable until numbered refs
const reinforced = reinforceMemory(
retained,
reinforcementSessionId(retained, dropped),
now,
);
const reinforcedEvent = reinforcementEvidence(retained, dropped, reinforced, reason);
if (reinforcedEvent) evidence.push(reinforcedEvent);
: "superseded_existing" as const;
if (reason === "superseded_existing") {
superseded.push(consolidationEvent(dropped, reason, reinforced));
superseded.push(consolidationEvent(dropped, reason, retained));
} else {
absorbed.push(consolidationEvent(dropped, reason, reinforced));
absorbed.push(consolidationEvent(dropped, reason, retained));
}
decisionDeduped.set(key, reinforced);
if (retained === entry) {
decisionDeduped.set(key, entry);
}
}
}
@@ -779,57 +595,13 @@ export function dedupeLongTermEntriesWithAccounting(entries: LongTermMemoryEntry
dropped: [],
absorbed,
superseded,
evidence,
};
}
function memoryEvidenceRef(memory: LongTermMemoryEntry): MemoryEvidenceRef {
return {
memoryId: memory.id,
memoryKeyHash: workspaceMemoryExactKey(memory),
identityKeyHash: workspaceMemoryIdentityKey(memory),
type: memory.type,
source: memory.source,
status: memory.status,
};
}
function reinforcementEvidence(
retained: LongTermMemoryEntry,
dropped: LongTermMemoryEntry,
reinforced: LongTermMemoryEntry,
reason: "absorbed_exact" | "absorbed_identity" | "superseded_existing",
): EvidenceEventInput | undefined {
if ((reinforced.reinforcementCount ?? 0) <= (retained.reinforcementCount ?? 0)) return undefined;
const duplicateReason = reason === "absorbed_identity" ? "duplicate_identity" : "duplicate_exact";
return {
type: "memory_reinforced",
phase: "reinforcement",
outcome: "reinforced",
memory: memoryEvidenceRef(reinforced),
relations: [
{ role: "reinforced", memory: memoryEvidenceRef(reinforced) },
{ role: "reinforced_by", memory: memoryEvidenceRef(dropped) },
],
reasonCodes: [duplicateReason, "reinforcement_window_allowed"],
textPreview: reinforced.text,
};
}
function reinforcementSessionId(retained: LongTermMemoryEntry, dropped: LongTermMemoryEntry): string {
return dropped.pendingOwnerSessionID ?? retained.pendingOwnerSessionID ?? "workspace-dedupe";
}
function compareLongTermMemoryForRetention(
a: LongTermMemoryEntry,
b: LongTermMemoryEntry,
now: number,
lastActivityAt?: string,
): number {
const strengthA = calculateRetentionStrength(a, now, lastActivityAt);
const strengthB = calculateRetentionStrength(b, now, lastActivityAt);
if (strengthB !== strengthA) return strengthB - strengthA;
function compareLongTermMemoryForRetention(a: LongTermMemoryEntry, b: LongTermMemoryEntry): number {
const pA = priority(a);
const pB = priority(b);
if (pB !== pA) return pB - pA;
const sourceDiff = sourcePriority(b.source) - sourcePriority(a.source);
if (sourceDiff !== 0) return sourceDiff;
const createdDiff = new Date(b.createdAt).getTime() - new Date(a.createdAt).getTime();
@@ -837,6 +609,18 @@ function compareLongTermMemoryForRetention(
return a.id.localeCompare(b.id);
}
function priority(entry: LongTermMemoryEntry): number {
const typeWeight = {
feedback: 400,
decision: 300,
project: 200,
reference: 100,
}[entry.type];
const sourceWeight = entry.source === "explicit" ? 1000 : 0;
return sourceWeight + typeWeight + entry.confidence * 10;
}
function wouldFit(
lines: string[],
nextLine: string,
@@ -847,49 +631,20 @@ function wouldFit(
}
export function renderWorkspaceMemory(store: WorkspaceMemoryStore): string {
return accountWorkspaceMemoryRender(store).prompt;
}
const active = enforceLongTermLimits(store.entries);
if (active.length === 0) return "";
export function accountWorkspaceMemoryRender(store: WorkspaceMemoryStore): WorkspaceMemoryRenderAccounting {
const now = Date.now();
const maxChars = Math.min(
store.limits.maxRenderedChars,
LONG_TERM_LIMITS.maxRenderedChars
);
const omitted: WorkspaceMemoryRenderAccounting["omitted"] = [];
const evidence: EvidenceEventInput[] = [];
for (const entry of store.entries) {
if (entry.status === "superseded") {
omitted.push({ memory: entry, reason: "superseded" });
}
}
const activeEntries = store.entries.filter(entry => entry.status !== "superseded");
const phase1 = activeEntries.map(entry => ({ ...entry, text: entry.text.slice(0, LONG_TERM_LIMITS.maxEntryTextChars) }));
const dedupeResult = dedupeLongTermEntriesWithAccounting(phase1);
const sorted = [...dedupeResult.kept].sort((a, b) => compareLongTermMemoryForRetention(a, b, now, store.lastActivityAt));
const typeCapResult = applyTypeMaxCapsWithOmissions(sorted);
for (const memory of typeCapResult.omitted) omitted.push({ memory, reason: "type_cap" });
const active = typeCapResult.kept.slice(0, LONG_TERM_LIMITS.maxEntries);
for (const memory of typeCapResult.kept.slice(LONG_TERM_LIMITS.maxEntries)) omitted.push({ memory, reason: "global_cap" });
if (active.length === 0) {
for (const item of omitted) evidence.push(renderEvidence(item.memory, "omitted", item.reason));
return { rendered: [], omitted, evidence, prompt: "" };
}
// If maxChars smaller than minimum envelope, return empty string
if (maxChars < MIN_ENVELOPE_LENGTH) {
for (const memory of active) omitted.push({ memory, reason: "empty_render_budget" });
for (const item of omitted) evidence.push(renderEvidence(item.memory, "omitted", item.reason));
return { rendered: [], omitted, evidence, prompt: "" };
}
if (maxChars < MIN_ENVELOPE_LENGTH) return "";
const lines: string[] = [
"Workspace memory (cross-session, verify if stale):",
];
const rendered: LongTermMemoryEntry[] = [];
for (const type of ["feedback", "project", "decision", "reference"] as const) {
const items = active.filter(entry => entry.type === type);
@@ -901,9 +656,6 @@ export function accountWorkspaceMemoryRender(store: WorkspaceMemoryStore): Works
const line = `- ${renderEntry(item)}`;
if ([...lines, ...sectionLines, line].join("\n").length <= maxChars) {
sectionLines.push(line);
rendered.push(item);
} else {
omitted.push({ memory: item, reason: "char_budget" });
}
}
@@ -912,26 +664,7 @@ export function accountWorkspaceMemoryRender(store: WorkspaceMemoryStore): Works
}
}
for (const memory of rendered) evidence.push(renderEvidence(memory, "rendered"));
for (const item of omitted) evidence.push(renderEvidence(item.memory, "omitted", item.reason));
return { rendered, omitted, evidence, prompt: lines.join("\n") };
}
function renderEvidence(
memory: LongTermMemoryEntry,
outcome: "rendered" | "omitted",
reason?: WorkspaceMemoryRenderAccounting["omitted"][number]["reason"],
): EvidenceEventInput {
return {
type: outcome === "rendered" ? "render_selected" : "render_omitted",
phase: "render",
outcome,
memory: memoryEvidenceRef(memory),
relations: [{ role: outcome === "rendered" ? "rendered" : "omitted", memory: memoryEvidenceRef(memory) }],
reasonCodes: outcome === "rendered" ? ["within_caps", "within_char_budget"] : [reason ?? "char_budget"],
textPreview: memory.text,
};
return lines.join("\n");
}
function renderEntry(entry: LongTermMemoryEntry): string {
-323
View File
@@ -1,323 +0,0 @@
import test from "node:test";
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { existsSync } from "node:fs";
import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { realpath } from "node:fs/promises";
import { tmpdir } from "node:os";
import {
EVIDENCE_LOG_LIMITS,
appendEvidenceEvent,
appendEvidenceEvents,
queryEvidenceEvents,
summarizeMemoryEvidence,
traceMemoryLifecycle,
type EvidenceEventInput,
type EvidenceEventV1,
} from "../src/evidence-log.ts";
import { workspaceEvidenceLogPath, workspaceKey } from "../src/paths.ts";
async function tempRoot(): Promise<string> {
return mkdtemp(join(tmpdir(), "opencode-evidence-log-"));
}
function eventInput(overrides: Partial<EvidenceEventInput> = {}): EvidenceEventInput {
return {
type: "promotion_promoted",
phase: "promotion",
outcome: "promoted",
reasonCodes: ["new_workspace_entry"],
memory: { memoryId: "mem-a", type: "decision", source: "explicit", status: "active" },
textPreview: "Use npm test before release",
...overrides,
};
}
async function readLog(root: string): Promise<string> {
return readFile(await workspaceEvidenceLogPath(root), "utf8");
}
function privacyHash(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 16);
}
async function workspaceRootHash(root: string): Promise<string> {
const resolved = await realpath(root).catch(() => root);
return privacyHash(resolved);
}
function manualEvent(rootKey: string, rootHash: string, id: string, createdAt: string): EvidenceEventV1 {
return {
version: 1,
eventId: id,
createdAt,
workspaceKey: rootKey,
workspaceRootHash: rootHash,
type: "render_selected",
phase: "render",
outcome: "rendered",
memory: { memoryId: id, type: "decision" },
reasonCodes: ["within_caps", "within_char_budget"],
};
}
test("appendEvidenceEvent writes versioned JSONL with workspace hashes", async () => {
const root = await tempRoot();
try {
const event = await appendEvidenceEvent(root, eventInput({ sessionHash: "session-1", messageHash: "message-1" }));
const raw = await readLog(root);
const stored = JSON.parse(raw.trim()) as EvidenceEventV1;
assert.equal(stored.version, 1);
assert.match(stored.eventId, /^evt_\d+_[a-z0-9]{8}$/);
assert.equal(stored.eventId, event.eventId);
assert.equal(stored.workspaceKey, await workspaceKey(root));
assert.equal(stored.workspaceRootHash, await workspaceRootHash(root));
assert.equal(stored.sessionHash, privacyHash("session-1"));
assert.equal(stored.messageHash, privacyHash("message-1"));
assert.equal(stored.type, "promotion_promoted");
assert.equal(stored.outcome, "promoted");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("appendEvidenceEvent redacts text previews before writing", async () => {
const root = await tempRoot();
try {
await appendEvidenceEvent(root, eventInput({
type: "extraction_candidate_rejected",
phase: "extraction",
outcome: "rejected",
reasonCodes: ["raw_error"],
textPreview: "password: sushi\nAdmin PIN 是 456123\nBearer abc.def.ghi\nThis candidate is rejected and should be short",
details: {
note: "password: sushi Admin PIN 是 456123 Bearer abc.def.ghi",
},
}));
const raw = await readLog(root);
const stored = JSON.parse(raw.trim()) as EvidenceEventV1;
assert.ok(!raw.includes("sushi"));
assert.ok(!raw.includes("456123"));
assert.ok(!raw.includes("abc.def.ghi"));
assert.ok(stored.textPreview?.includes("[REDACTED]"));
assert.ok((stored.textPreview?.length ?? 0) <= 80);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("queryEvidenceEvents filters by type outcome and memory id", async () => {
const root = await tempRoot();
try {
await appendEvidenceEvents(root, [
eventInput({ type: "promotion_promoted", outcome: "promoted", memory: { memoryId: "mem-a" } }),
eventInput({ type: "render_omitted", phase: "render", outcome: "omitted", reasonCodes: ["type_cap"], memory: { memoryId: "mem-a" } }),
eventInput({ type: "render_omitted", phase: "render", outcome: "omitted", reasonCodes: ["global_cap"], memory: { memoryId: "mem-b" } }),
]);
const result = await queryEvidenceEvents(root, {
types: ["render_omitted"],
outcomes: ["omitted"],
memoryId: "mem-a",
});
assert.equal(result.length, 1);
assert.equal(result[0].type, "render_omitted");
assert.equal(result[0].memory?.memoryId, "mem-a");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory_removed_capacity event round-trips through append and query", async () => {
const root = await tempRoot();
try {
await appendEvidenceEvent(root, eventInput({
type: "memory_removed_capacity",
phase: "storage",
outcome: "removed",
reasonCodes: ["global_cap"],
memory: { memoryId: "removed-memory", type: "reference", source: "compaction", status: "active" },
relations: [{ role: "removed", memory: { memoryId: "removed-memory", type: "reference", source: "compaction", status: "active" } }],
details: {
type: "reference",
globalCap: 28,
retentionClock: Date.UTC(2026, 4, 1),
createdAt: "2026-05-01T00:00:00.000Z",
source: "compaction",
},
}));
const result = await queryEvidenceEvents(root, {
types: ["memory_removed_capacity"],
phases: ["storage"],
outcomes: ["removed"],
memoryId: "removed-memory",
});
assert.equal(result.length, 1);
assert.equal(result[0].type, "memory_removed_capacity");
assert.equal(result[0].phase, "storage");
assert.equal(result[0].outcome, "removed");
assert.deepEqual(result[0].details, {
type: "reference",
globalCap: 28,
retentionClock: Date.UTC(2026, 4, 1),
createdAt: "2026-05-01T00:00:00.000Z",
source: "compaction",
});
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory_migration_superseded event round-trips through append and query", async () => {
const root = await tempRoot();
try {
await appendEvidenceEvent(root, eventInput({
type: "memory_migration_superseded",
phase: "storage",
outcome: "superseded",
reasonCodes: ["migration:quality_cleanup", "quality:progress_snapshot"],
memory: { memoryId: "migrated-memory", type: "project", source: "compaction", status: "superseded" },
relations: [{ role: "superseded", memory: { memoryId: "migrated-memory", type: "project", source: "compaction", status: "superseded" } }],
details: {
migrationId: "2026-04-28-quality-cleanup",
type: "project",
source: "compaction",
},
}));
const result = await queryEvidenceEvents(root, {
types: ["memory_migration_superseded"],
phases: ["storage"],
outcomes: ["superseded"],
memoryId: "migrated-memory",
});
assert.equal(result.length, 1);
assert.equal(result[0].type, "memory_migration_superseded");
assert.equal(result[0].phase, "storage");
assert.equal(result[0].outcome, "superseded");
assert.ok(result[0].reasonCodes.includes("migration:quality_cleanup"));
assert.equal(result[0].memory?.status, "superseded");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("queryEvidenceEvents supports newestFirst and limit", async () => {
const root = await tempRoot();
try {
const events = await appendEvidenceEvents(root, [
eventInput({ memory: { memoryId: "oldest" } }),
eventInput({ memory: { memoryId: "middle" } }),
eventInput({ memory: { memoryId: "newest" } }),
]);
const result = await queryEvidenceEvents(root, { newestFirst: true, limit: 2 });
assert.deepEqual(result.map(event => event.eventId), [events[2].eventId, events[1].eventId]);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("appendEvidenceEvent returns a record when appendFile fails", async () => {
const root = await tempRoot();
try {
const path = await workspaceEvidenceLogPath(root);
await mkdir(path, { recursive: true });
const event = await appendEvidenceEvent(root, eventInput());
assert.equal(event.version, 1);
assert.match(event.eventId, /^evt_\d+_[a-z0-9]{8}$/);
assert.equal(event.workspaceKey, await workspaceKey(root));
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("evidence log pruning drops old events, caps count, and quarantines invalid lines", async () => {
const root = await tempRoot();
try {
const path = await workspaceEvidenceLogPath(root);
await mkdir(dirname(path), { recursive: true });
const rootKey = await workspaceKey(root);
const rootHash = await workspaceRootHash(root);
const old = manualEvent(rootKey, rootHash, "old-event", new Date(Date.now() - 91 * 24 * 60 * 60 * 1000).toISOString());
const recentEvents = Array.from({ length: EVIDENCE_LOG_LIMITS.maxEventsPerWorkspace + 1 }, (_, i) =>
manualEvent(rootKey, rootHash, `recent-${i}`, new Date(Date.now() - (EVIDENCE_LOG_LIMITS.maxEventsPerWorkspace - i) * 1000).toISOString())
);
await writeFile(path, [
JSON.stringify(old),
"{not valid json",
...recentEvents.map(event => JSON.stringify(event)),
].join("\n") + "\n", "utf8");
const appended = await appendEvidenceEvents(root, Array.from({ length: EVIDENCE_LOG_LIMITS.pruneEveryAppendCount }, (_, i) =>
eventInput({ memory: { memoryId: `appended-${i}` }, reasonCodes: ["new_workspace_entry"] })
));
const events = await queryEvidenceEvents(root);
const memoryIds = new Set(events.map(event => event.memory?.memoryId));
const files = await readdir(dirname(path));
assert.ok(events.length <= EVIDENCE_LOG_LIMITS.maxEventsPerWorkspace);
assert.equal(memoryIds.has("old-event"), false);
assert.equal(memoryIds.has("recent-0"), false, "oldest events over the count cap should be pruned");
assert.equal(memoryIds.has(appended.at(-1)?.memory?.memoryId), true);
assert.ok(files.some(file => file.startsWith("events.jsonl.corrupt-lines-")));
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory evidence summary and lifecycle trace derive latest status", async () => {
const root = await tempRoot();
try {
await appendEvidenceEvents(root, [
eventInput({ type: "extraction_candidate_accepted", phase: "extraction", outcome: "accepted", reasonCodes: ["quality_gate_passed"], memory: { memoryId: "mem-life", memoryKeyHash: "raw-key" } }),
eventInput({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", reasonCodes: ["new_workspace_entry"], memory: { memoryId: "mem-life", memoryKeyHash: "raw-key" } }),
eventInput({ type: "memory_reinforced", phase: "reinforcement", outcome: "reinforced", reasonCodes: ["duplicate_exact"], relations: [{ role: "reinforced", memory: { memoryId: "mem-life", memoryKeyHash: "raw-key" } }] }),
eventInput({ type: "render_selected", phase: "render", outcome: "rendered", reasonCodes: ["within_caps", "within_char_budget"], memory: { memoryId: "mem-life", memoryKeyHash: "raw-key" } }),
]);
const summary = await summarizeMemoryEvidence(root, { memoryId: "mem-life" });
const trace = await traceMemoryLifecycle(root, { memoryId: "mem-life" });
assert.equal(summary.latestOutcome, "rendered");
assert.equal(summary.latestRenderStatus, "rendered");
assert.ok(summary.reasonCodes.includes("duplicate_exact"));
assert.equal(trace.currentStatus, "rendered");
assert.ok(trace.acceptedBy);
assert.ok(trace.promotedBy);
assert.equal(trace.reinforcedBy.length, 1);
assert.equal(trace.latestRender?.type, "render_selected");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("evidence relation roles reject sublimation placeholders at compile-time surface", () => {
const allowedRoles: Array<NonNullable<EvidenceEventInput["relations"]>[number]["role"]> = [
"candidate",
"pending",
"promoted",
"retained",
"absorbed",
"superseded",
"superseded_by",
"reinforced",
"reinforced_by",
"rendered",
"omitted",
"removed",
];
assert.equal(allowedRoles.includes("candidate"), true);
});
+1 -94
View File
@@ -3,13 +3,7 @@ import assert from "node:assert/strict";
import { mkdtemp, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import {
extractErrorsFromBash,
extractExplicitMemories,
extractExplicitMemoriesWithEvidence,
parseWorkspaceMemoryCandidates,
parseWorkspaceMemoryCandidatesWithEvidence,
} from "../src/extractors.ts";
import { extractErrorsFromBash, extractExplicitMemories, parseWorkspaceMemoryCandidates } from "../src/extractors.ts";
async function waitForFile(path: string, attempts = 20): Promise<string> {
let lastError: unknown;
@@ -97,13 +91,9 @@ test("extractExplicitMemories does not treat always as memory trigger", () => {
});
test("extractExplicitMemories still captures going forward", () => {
const before = Date.now();
const items = extractExplicitMemories("going forward: use pnpm instead of npm");
const after = Date.now();
assert.equal(items.length, 1);
assert.match(items[0].text, /pnpm/);
assert.ok(typeof items[0].retentionClock === "number");
assert.ok(items[0].retentionClock >= before && items[0].retentionClock <= after);
});
test("extractExplicitMemories captures from now on", () => {
@@ -151,20 +141,6 @@ test("extractExplicitMemories captures multiple memories in same message", () =>
assert.equal(items.length, 2);
});
test("explicit memory extraction returns detected and ignored evidence", () => {
const result = extractExplicitMemoriesWithEvidence([
"remember this: Prefer deterministic tests.",
"don't remember this: temporary password: sushi",
"remember this: later",
].join("\n"));
assert.equal(result.entries.length, 1);
assert.ok(result.evidence.some(event => event.type === "explicit_memory_detected"));
assert.ok(result.evidence.some(event => event.type === "explicit_memory_ignored" && event.reasonCodes.includes("negated_request")));
assert.ok(result.evidence.some(event => event.type === "explicit_memory_ignored" && event.reasonCodes.includes("deferral")));
assert.equal(JSON.stringify(result.evidence).includes("sushi"), false);
});
// ============================================
// Task 7: Compaction quality gate tests
// ============================================
@@ -196,41 +172,6 @@ test("parseWorkspaceMemoryCandidates rejects raw error", () => {
assert.equal(items.length, 0);
});
test("compaction accepted candidate returns privacy-safe extraction evidence", () => {
const summary = `
Memory candidates:
- [decision] Use accounting evidence events to explain promoted memories in diagnostics.
`;
const result = parseWorkspaceMemoryCandidatesWithEvidence(summary);
assert.equal(result.entries.length, 1);
assert.equal(result.evidence.length, 1);
assert.equal(result.evidence[0].type, "extraction_candidate_accepted");
assert.ok(result.evidence[0].reasonCodes.includes("quality_gate_passed"));
assert.ok(result.evidence[0].reasonCodes.includes("valid_candidate_format"));
assert.match(result.evidence[0].textPreview ?? "", /accounting evidence events/);
});
test("compaction rejected candidate returns rejection evidence without secrets", () => {
const summary = `
Memory candidates:
- [feedback] password: sushi Admin PIN 是 456123 Bearer abc.def.ghi TypeError: Cannot read property x
`;
const result = parseWorkspaceMemoryCandidatesWithEvidence(summary);
const raw = JSON.stringify(result.evidence);
assert.equal(result.entries.length, 0);
assert.equal(result.evidence.length, 1);
assert.equal(result.evidence[0].type, "extraction_candidate_rejected");
assert.ok(result.evidence[0].reasonCodes.length > 0);
assert.equal(raw.includes("sushi"), false);
assert.equal(raw.includes("456123"), false);
assert.equal(raw.includes("abc.def.ghi"), false);
assert.ok((result.evidence[0].textPreview?.length ?? 0) <= 80);
});
test("parseWorkspaceMemoryCandidates rejects stack trace", () => {
const summary = `
## Memory Candidates
@@ -259,18 +200,14 @@ test("parseWorkspaceMemoryCandidates rejects path-heavy facts", () => {
});
test("parseWorkspaceMemoryCandidates accepts valid decision", () => {
const before = Date.now();
const summary = `
## Memory Candidates
- [decision] Use pnpm instead of npm for package management
`;
const items = parseWorkspaceMemoryCandidates(summary);
const after = Date.now();
assert.equal(items.length, 1);
assert.equal(items[0].type, "decision");
assert.match(items[0].text, /pnpm/);
assert.ok(typeof items[0].retentionClock === "number");
assert.ok(items[0].retentionClock >= before && items[0].retentionClock <= after);
});
test("parseWorkspaceMemoryCandidates accepts valid project info", () => {
@@ -387,36 +324,6 @@ Memory candidates:
}
});
test("new rejection records include workspaceKey and workspaceRootHash when provided", async () => {
const dataHome = await mkdtemp(join(tmpdir(), "wm-extraction-scope-data-"));
const previousXdgDataHome = process.env.XDG_DATA_HOME;
process.env.XDG_DATA_HOME = dataHome;
try {
const summary = `
Memory candidates:
- feedback Wave 1 completed successfully and all tests passed
`;
const result = parseWorkspaceMemoryCandidatesWithEvidence(summary, {
workspaceKey: "testkey1234567",
workspaceRootHash: "abcdef123456",
});
assert.equal(result.entries.length, 0);
const logPath = join(dataHome, "opencode-working-memory", "extraction-rejections.jsonl");
const lines = (await waitForFile(logPath)).trim().split("\n");
assert.equal(lines.length, 1);
const event = JSON.parse(lines[0]);
assert.equal(event.workspaceKey, "testkey1234567");
assert.equal(event.workspaceRootHash, "abcdef123456");
} finally {
if (previousXdgDataHome === undefined) delete process.env.XDG_DATA_HOME;
else process.env.XDG_DATA_HOME = previousXdgDataHome;
await rm(dataHome, { recursive: true, force: true });
}
});
test("parseWorkspaceMemoryCandidates redacts secrets in extraction rejection log", async () => {
const dataHome = await mkdtemp(join(tmpdir(), "wm-extraction-redact-data-"));
const previousXdgDataHome = process.env.XDG_DATA_HOME;
-135
View File
@@ -1,135 +0,0 @@
import test from "node:test";
import assert from "node:assert/strict";
import type { EvidenceEventType, EvidenceEventV1, EvidenceOutcome, EvidencePhase } from "../src/evidence-log.ts";
import type { LongTermMemoryEntry } from "../src/types.ts";
import type { MemoryInspectionReadModel } from "../scripts/memory-diag/types.ts";
import { CliInputError, normalizeRejection, rejectionQualitySummary, sinceCutoff } from "../scripts/memory-diag/rejections-model.ts";
import { coverageRows, disappearanceRows } from "../scripts/memory-diag/inspection-model.ts";
import { groupEvidenceByMemoryId } from "../scripts/memory-diag/evidence-model.ts";
import { statusFromTraceEvent } from "../scripts/memory-diag/trace-model.ts";
function entry(id: string, type: LongTermMemoryEntry["type"]): LongTermMemoryEntry {
const now = new Date("2026-01-01T00:00:00.000Z").toISOString();
return {
id,
type,
text: `${id} text`,
source: "compaction",
confidence: 0.75,
status: "active",
createdAt: now,
updatedAt: now,
};
}
function event(overrides: Partial<EvidenceEventV1> & { type: EvidenceEventType; phase: EvidencePhase; outcome: EvidenceOutcome }): EvidenceEventV1 {
return {
version: 1,
eventId: `evt-${overrides.type}-${Math.random()}`,
createdAt: new Date("2026-01-01T00:00:00.000Z").toISOString(),
workspaceKey: "workspace-key",
workspaceRootHash: "workspace-root-hash",
reasonCodes: [],
...overrides,
};
}
function model(entries: LongTermMemoryEntry[], events: EvidenceEventV1[]): MemoryInspectionReadModel {
return {
store: {
version: 1,
workspace: { root: "/tmp/workspace", key: "workspace-key" },
limits: { maxRenderedChars: 24_000, maxEntries: 28 },
entries,
migrations: [],
updatedAt: new Date("2026-01-01T00:00:00.000Z").toISOString(),
},
pending: { version: 1, workspace: { root: "", key: "" }, entries: [], updatedAt: new Date(0).toISOString() },
evidenceEvents: events,
rejectionRecords: [],
currentById: new Map(entries.map(memory => [memory.id, memory])),
evidenceByMemoryId: groupEvidenceByMemoryId(events),
};
}
test("normalizeRejection infers origins from source", () => {
assert.equal(normalizeRejection({ source: "compaction", text: "a", reasons: ["bad_decision"] })?.origin, "compaction_candidate");
assert.equal(normalizeRejection({ source: "explicit", text: "a", reasons: ["bad_feedback"] })?.origin, "explicit_trigger");
assert.equal(normalizeRejection({ source: "manual", text: "a", reasons: ["bad_feedback"] })?.origin, "manual");
assert.equal(normalizeRejection({ source: "unknown-source", text: "a", reasons: ["bad_feedback"] })?.origin, "unknown");
});
test("sinceCutoff accepts relative durations and ISO timestamps", () => {
const now = new Date("2026-01-15T12:00:00.000Z").getTime();
assert.equal(sinceCutoff("14d", now), now - 14 * 86_400_000);
assert.equal(sinceCutoff("3h", now), now - 3 * 3_600_000);
assert.equal(sinceCutoff("30m", now), now - 30 * 60_000);
assert.equal(sinceCutoff("2026-01-01T00:00:00.000Z", now), new Date("2026-01-01T00:00:00.000Z").getTime());
assert.throws(() => sinceCutoff("forever", now), (error: unknown) => {
assert.ok(error instanceof CliInputError);
assert.equal((error as Error).message, "Invalid --since value: forever");
return true;
});
});
test("rejectionQualitySummary keeps architecture-like false-positive grouping", () => {
const records = [
normalizeRejection({ type: "decision", source: "compaction", text: "Retention scoring model uses evidence caps to avoid normalization drift", reasons: ["bad_decision"] }),
normalizeRejection({ type: "decision", source: "compaction", text: "Implemented phase 2 and updated tests", reasons: ["bad_decision"] }),
normalizeRejection({ type: "decision", source: "compaction", text: "Maybe useful", reasons: ["bad_decision"] }),
].filter(record => record !== null);
const summary = rejectionQualitySummary(records);
assert.equal(summary.totalRecords, 3);
assert.equal(summary.possibleFalsePositiveGroups.architecture_like_possible_false_positive.count, 1);
assert.equal(summary.possibleFalsePositiveGroups.clearly_garbage.count, 1);
assert.equal(summary.possibleFalsePositiveGroups.ambiguous.count, 1);
});
test("coverageRows classifies current and historical memory evidence", () => {
const entries = [entry("mem-full", "feedback"), entry("mem-render-only", "decision"), entry("mem-no-evidence", "project")];
const events = [
event({ type: "extraction_candidate_accepted", phase: "extraction", outcome: "accepted", memory: { memoryId: "mem-full", type: "feedback", source: "compaction" } }),
event({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "mem-full", type: "feedback", source: "compaction" } }),
event({ type: "render_selected", phase: "render", outcome: "rendered", memory: { memoryId: "mem-render-only", type: "decision", source: "compaction" } }),
event({ type: "memory_removed_capacity", phase: "storage", outcome: "removed", memory: { memoryId: "historical-cap", type: "project", source: "compaction" }, reasonCodes: ["global_cap"] }),
event({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "historical-unknown", type: "reference", source: "compaction" } }),
];
const rows = coverageRows(model(entries, events), true);
const byId = new Map(rows.map(row => [row.id, row.class]));
assert.equal(byId.get("mem-full"), "full_lifecycle");
assert.equal(byId.get("mem-render-only"), "render_only");
assert.equal(byId.get("mem-no-evidence"), "no_evidence");
assert.equal(byId.get("historical-cap"), "historical_absent_with_reason");
assert.equal(byId.get("historical-unknown"), "historical_absent_unknown_reason");
});
test("disappearanceRows surfaces terminal capacity evidence", () => {
const events = [
event({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "capacity-loser", type: "decision", source: "compaction" } }),
event({ type: "memory_removed_capacity", phase: "storage", outcome: "removed", memory: { memoryId: "capacity-loser", type: "decision", source: "compaction" }, reasonCodes: ["type_cap"] }),
];
const rows = disappearanceRows(model([], events));
assert.equal(rows.length, 1);
assert.equal(rows[0].id, "capacity-loser");
assert.equal(rows[0].classification, "historical_absent_with_reason");
assert.equal(rows[0].terminalType, "memory_removed_capacity");
assert.deepEqual(rows[0].reasonCodes, ["type_cap"]);
});
test("statusFromTraceEvent maps lifecycle events", () => {
assert.equal(statusFromTraceEvent(undefined), "unknown");
assert.equal(statusFromTraceEvent(event({ type: "render_selected", phase: "render", outcome: "rendered" })), "rendered");
assert.equal(statusFromTraceEvent(event({ type: "render_omitted", phase: "render", outcome: "omitted", reasonCodes: ["type_cap"] })), "omitted_type_cap");
assert.equal(statusFromTraceEvent(event({ type: "promotion_absorbed_exact", phase: "promotion", outcome: "absorbed" })), "omitted_absorbed_duplicate");
assert.equal(statusFromTraceEvent(event({ type: "promotion_retry_scheduled", phase: "promotion", outcome: "retried" })), "pending_retry");
assert.equal(statusFromTraceEvent(event({ type: "promotion_retry_exhausted", phase: "promotion", outcome: "exhausted" })), "pending_rejected_capacity");
assert.equal(statusFromTraceEvent(event({ type: "storage_corrupt_json_quarantined", phase: "storage", outcome: "quarantined" })), "quarantined_corrupt_store");
assert.equal(statusFromTraceEvent(event({ type: "promotion_superseded", phase: "promotion", outcome: "superseded" })), "omitted_superseded");
});
-105
View File
@@ -1,105 +0,0 @@
import test from "node:test";
import assert from "node:assert/strict";
import { parseArgs } from "../scripts/memory-diag/cli.ts";
test("help returns usage without exposing hidden or removed commands", () => {
const parsed = parseArgs(["--help"]);
assert.equal(parsed.ok, true);
assert.equal("help" in parsed && parsed.help, true);
assert.match(parsed.usage, /Usage:/);
assert.match(parsed.usage, /memory-diag \[status\]/);
for (const command of ["health", "quality", "rejections", "disappearances", "trace", "coverage", "audit"]) {
assert.doesNotMatch(parsed.usage, new RegExp(command));
}
});
test("status defaults when no subcommand", () => {
const parsed = parseArgs([]);
assert.equal(parsed.ok, true);
assert.equal("command" in parsed && parsed.command, "status");
assert.deepEqual("options" in parsed && parsed.options, { raw: false, positional: [] });
});
test("unknown command returns usage error", () => {
const parsed = parseArgs(["unknown"]);
assert.equal(parsed.ok, false);
if (parsed.ok) return;
assert.equal(parsed.message, "Unknown subcommand: unknown");
assert.equal(parsed.exitCode, 1);
assert.match(parsed.usage, /Usage:/);
});
test("removed legacy aliases are ordinary unknown subcommands", () => {
for (const command of ["health", "quality", "rejections", "disappearances", "trace"]) {
const parsed = parseArgs([command]);
assert.equal(parsed.ok, false, command);
if (parsed.ok) continue;
assert.equal(parsed.message, `Unknown subcommand: ${command}`);
assert.match(parsed.usage, /Usage:/);
}
});
test("hidden maintainer commands are accepted with neutral notices", () => {
const coverage = parseArgs(["coverage"]);
assert.equal(coverage.ok, true);
assert.equal("command" in coverage && coverage.command, "coverage");
assert.equal("deprecationNotice" in coverage && coverage.deprecationNotice, "Note: 'coverage' is a maintainer-only diagnostic and is not part of the public CLI surface.");
const audit = parseArgs(["audit"]);
assert.equal(audit.ok, true);
assert.equal("command" in audit && audit.command, "audit");
assert.equal("deprecationNotice" in audit && audit.deprecationNotice, "Note: 'audit' is a maintainer-only diagnostic and is not part of the public CLI surface.");
});
test("current command flag validation messages are preserved", () => {
const cases: Array<{ args: string[]; message: string }> = [
{ args: ["status", "--all"], message: "status does not accept --all" },
{ args: ["coverage", "--all"], message: "coverage does not accept --all" },
{ args: ["missing", "--all"], message: "missing does not accept --all" },
{ args: ["rejected", "--all"], message: "rejected does not accept --all" },
{ args: ["audit", "--workspace", "/tmp/workspace"], message: "audit does not accept --all or --workspace" },
{ args: ["explain", "--all"], message: "explain does not accept --all" },
{ args: ["status", "--since", "forever"], message: "status does not accept rejection filters" },
{ args: ["status", "--reason", "bad_decision"], message: "status does not accept rejection filters" },
{ args: ["status", "--quality"], message: "Unknown option: --quality" },
{ args: ["rejected", "--unique"], message: "Unknown option: --unique" },
];
for (const item of cases) {
const parsed = parseArgs(item.args);
assert.equal(parsed.ok, false, item.args.join(" "));
if (parsed.ok) continue;
assert.equal(parsed.message, item.message);
assert.match(parsed.usage, /Usage:/);
}
});
test("rejected invalid since value returns current error", () => {
const parsed = parseArgs(["rejected", "--since", "forever"]);
assert.equal(parsed.ok, false);
if (parsed.ok) return;
assert.equal(parsed.message, "Invalid --since value: forever");
assert.match(parsed.usage, /Usage:/);
});
test("explain accepts positional memory id", () => {
const parsed = parseArgs(["explain", "mem-1", "--workspace", "/tmp/workspace"]);
assert.equal(parsed.ok, true);
assert.equal("command" in parsed && parsed.command, "explain");
assert.equal("options" in parsed && parsed.options.memory, "mem-1");
assert.deepEqual("options" in parsed && parsed.options.positional, ["mem-1"]);
});
test("explain with both positional and memory flag errors", () => {
const parsed = parseArgs(["explain", "mem-1", "--memory", "mem-2"]);
assert.equal(parsed.ok, false);
if (parsed.ok) return;
assert.equal(parsed.message, "Use either explain <memory-id> or --memory, not both");
});
-110
View File
@@ -1,110 +0,0 @@
import test from "node:test";
import assert from "node:assert/strict";
import type { EvidenceEventType, EvidenceEventV1, EvidenceOutcome, EvidencePhase } from "../src/evidence-log.ts";
import type { MemoryInspectionReadModel, WorkspaceDiagSnapshot } from "../scripts/memory-diag/types.ts";
import { formatCoverage } from "../scripts/memory-diag/formatters/coverage.ts";
import { formatMigrationAudit } from "../scripts/memory-diag/formatters/audit.ts";
import { formatExplain } from "../scripts/memory-diag/formatters/explain.ts";
import { buildMissingJSON, formatMissing } from "../scripts/memory-diag/formatters/missing.ts";
import { formatTrace } from "../scripts/memory-diag/formatters/trace.ts";
function emptyInspectionModel(): MemoryInspectionReadModel {
return {
store: {
version: 1,
workspace: { root: "/tmp/workspace", key: "workspace-key" },
limits: { maxRenderedChars: 24_000, maxEntries: 28 },
entries: [],
migrations: [],
updatedAt: new Date("2026-01-01T00:00:00.000Z").toISOString(),
},
pending: { version: 1, workspace: { root: "", key: "" }, entries: [], updatedAt: new Date(0).toISOString() },
evidenceEvents: [],
rejectionRecords: [],
currentById: new Map(),
evidenceByMemoryId: new Map(),
};
}
function emptySnapshot(): WorkspaceDiagSnapshot {
return {
store: emptyInspectionModel().store,
journal: emptyInspectionModel().pending,
retention: { sorted: [], rendered: [], typeCapped: [], globalCapped: [] },
memories: [],
recentEvents: [],
allEvents: [],
summary: {
storedActive: 0,
rendered: 0,
pending: 0,
rejectedLast7Days: 0,
corruptStoresQuarantinedLast30Days: 0,
},
};
}
function event(overrides: Partial<EvidenceEventV1> & { type: EvidenceEventType; phase: EvidencePhase; outcome: EvidenceOutcome }): EvidenceEventV1 {
return {
version: 1,
eventId: `evt-${overrides.type}`,
createdAt: new Date("2026-01-01T00:00:00.000Z").toISOString(),
workspaceKey: "workspace-key",
workspaceRootHash: "workspace-root-hash",
reasonCodes: [],
...overrides,
};
}
test("coverage formatter includes class counts section", () => {
const output = formatCoverage([]);
assert.match(output, /Class counts:/);
assert.match(output, /Per-memory rows:\n \(none\)/);
});
test("missing formatter verbose output preserves disappearance details", () => {
const output = formatMissing([], { verbose: true });
assert.match(output, /No missing memories found\./);
});
test("missing JSON includes disappearance rows and summary", () => {
const rows = [{
id: "historical-1",
classification: "historical_absent_unknown_reason" as const,
terminalType: "unknown" as const,
reasonCodes: [],
events: [event({ type: "extraction_candidate_accepted", phase: "extraction", outcome: "accepted" })],
event: undefined,
}];
const output = buildMissingJSON(rows, { generatedAt: "2026-01-01T00:00:00.000Z" });
assert.equal(output.version, 1);
assert.deepEqual(output.summary, { total: 1, explained: 0, needsReview: 1 });
assert.deepEqual((output.disappearances as Array<{ id: string }>)[0]?.id, "historical-1");
});
test("trace formatter includes lifecycle section", () => {
const output = formatTrace("mem-1", emptySnapshot(), {
events: [event({ type: "render_selected", phase: "render", outcome: "rendered", memory: { memoryId: "mem-1", type: "feedback", source: "explicit" } })],
});
assert.match(output, /Lifecycle:/);
assert.match(output, /evt-render_selected render_selected/);
});
test("audit formatter preserves no-log output", () => {
const output = formatMigrationAudit([], { raw: false });
assert.match(output, /Migration audit report/);
assert.match(output, /No migration logs found\./);
});
test("explain formatter preserves no-memory output", () => {
const output = formatExplain(emptySnapshot());
assert.match(output, /Workspace memory explainability/);
assert.match(output, /No memories found\./);
});
-54
View File
@@ -1,54 +0,0 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { canonicalMemoryText, cleanText, countBy, sortedCounts, truncate } from "../scripts/memory-diag/text.ts";
import { readJSONLFile } from "../scripts/memory-diag/io.ts";
test("cleanText redacts credentials and absolute paths unless raw", () => {
const text = "Use password: sushi and api_key=abc123 in /Users/alice/project/config.json";
const cleaned = cleanText(text, false);
assert.match(cleaned, /password: \[REDACTED\]/);
assert.match(cleaned, /api_key=\[REDACTED\]/);
assert.match(cleaned, /<path>/);
assert.doesNotMatch(cleaned, /sushi/);
assert.doesNotMatch(cleaned, /\/Users\/alice/);
assert.equal(cleanText(text, true), text);
});
test("truncate collapses whitespace and applies ellipsis at max length", () => {
assert.equal(truncate(" hello\n\tworld "), "hello world");
assert.equal(truncate("abcdef", 5), "abcd…");
});
test("canonicalMemoryText normalizes punctuation and case", () => {
assert.equal(canonicalMemoryText("Hello, WORLD!!! Path?"), "hello world path");
});
test("sortedCounts sorts by count descending then key ascending", () => {
const counts = new Map<string, number>([["b", 2], ["a", 2], ["c", 3]]);
assert.deepEqual(sortedCounts(counts), [["c", 3], ["a", 2], ["b", 2]]);
});
test("countBy counts string items", () => {
assert.deepEqual([...countBy(["beta", "alpha", "beta"]).entries()], [["beta", 2], ["alpha", 1]]);
});
test("readJSONLFile returns valid records and invalid line count", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-utils-"));
try {
const path = join(root, "records.jsonl");
await writeFile(path, '{"id":"one"}\nnot-json\n\n{"id":"two"}\n', "utf8");
const result = await readJSONLFile<{ id: string }>(path);
assert.deepEqual(result.records, [{ id: "one" }, { id: "two" }]);
assert.equal(result.invalidLines, 1);
} finally {
await rm(root, { recursive: true, force: true });
}
});
-224
View File
@@ -1,224 +0,0 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { appendEvidenceEvents, queryEvidenceEvents, summarizeMemoryEvidence, type EvidenceEventInput, type EvidenceEventV1 } from "../src/evidence-log.ts";
import { groupEvidenceByMemoryId } from "../scripts/memory-diag/evidence-model.ts";
import { retentionCandidatesForDiag } from "../scripts/memory-diag/retention-model.ts";
import { buildMemoryDiagJSON, memoryDiagJSONFromSnapshot, normalizedJournal, normalizedStore, snapshotForOptions } from "../scripts/memory-diag/workspace-snapshot.ts";
import { workspaceKey, workspaceMemoryPath, workspacePendingJournalPath } from "../src/paths.ts";
import { LONG_TERM_LIMITS, type LongTermMemoryEntry, type PendingMemoryJournalStore, type WorkspaceMemoryStore } from "../src/types.ts";
function entry(id: string, text: string, type: LongTermMemoryEntry["type"]): LongTermMemoryEntry {
const now = new Date("2026-01-01T00:00:00.000Z").toISOString();
return {
id,
type,
text,
source: "compaction",
confidence: 0.75,
status: "active",
createdAt: now,
updatedAt: now,
retentionClock: new Date(now).getTime(),
};
}
function evidence(overrides: Partial<EvidenceEventInput>): EvidenceEventInput {
return {
type: "promotion_promoted",
phase: "promotion",
outcome: "promoted",
memory: { memoryId: "mem-active", type: "decision", source: "compaction", status: "active" },
reasonCodes: ["new_workspace_entry"],
...overrides,
};
}
function groupedEvidenceSummary(grouped: Map<string, EvidenceEventV1[]>, memoryId: string): { eventIds: string[]; reasonCodes: string[] } {
const events = grouped.get(memoryId) ?? [];
const reasonCodes = new Set<string>();
for (const event of events) {
for (const reason of event.reasonCodes) reasonCodes.add(reason);
}
return {
eventIds: events.map(event => event.eventId),
reasonCodes: [...reasonCodes],
};
}
async function writeWorkspaceStore(root: string, entries: LongTermMemoryEntry[]): Promise<void> {
const key = await workspaceKey(root);
const path = await workspaceMemoryPath(root);
const store: WorkspaceMemoryStore = {
version: 1,
workspace: { root, key },
limits: { maxRenderedChars: LONG_TERM_LIMITS.maxRenderedChars, maxEntries: LONG_TERM_LIMITS.maxEntries },
entries,
migrations: [],
updatedAt: new Date("2026-01-01T00:00:00.000Z").toISOString(),
};
await mkdir(dirname(path), { recursive: true });
await writeFile(path, JSON.stringify(store, null, 2), "utf8");
}
async function writePendingJournal(root: string, entries: LongTermMemoryEntry[]): Promise<void> {
const key = await workspaceKey(root);
const path = await workspacePendingJournalPath(root);
const store: PendingMemoryJournalStore = {
version: 1,
workspace: { root, key },
entries,
updatedAt: new Date("2026-01-01T00:00:00.000Z").toISOString(),
};
await mkdir(dirname(path), { recursive: true });
await writeFile(path, JSON.stringify(store, null, 2), "utf8");
}
test("normalizedStore returns an empty store with limits and empty arrays", () => {
const store = normalizedStore(null, "/tmp/example-workspace", "workspace-key");
assert.equal(store.version, 1);
assert.deepEqual(store.workspace, { root: "/tmp/example-workspace", key: "workspace-key" });
assert.deepEqual(store.limits, { maxRenderedChars: LONG_TERM_LIMITS.maxRenderedChars, maxEntries: LONG_TERM_LIMITS.maxEntries });
assert.deepEqual(store.entries, []);
assert.deepEqual(store.migrations, []);
});
test("normalizedJournal returns an empty journal", () => {
const journal = normalizedJournal(null);
assert.equal(journal.version, 1);
assert.deepEqual(journal.workspace, { root: "", key: "" });
assert.deepEqual(journal.entries, []);
assert.equal(journal.updatedAt, new Date(0).toISOString());
});
test("retentionCandidatesForDiag separates rendered, type-capped, and global-capped entries", () => {
const entries: LongTermMemoryEntry[] = [
...Array.from({ length: 11 }, (_, i) => entry(`feedback-${String(i).padStart(2, "0")}`, `Feedback memory ${i}`, "feedback")),
...Array.from({ length: 10 }, (_, i) => entry(`decision-${String(i).padStart(2, "0")}`, `Decision memory ${i}`, "decision")),
...Array.from({ length: 8 }, (_, i) => entry(`project-${String(i).padStart(2, "0")}`, `Project memory ${i}`, "project")),
...Array.from({ length: 6 }, (_, i) => entry(`reference-${String(i).padStart(2, "0")}`, `Reference memory ${i}`, "reference")),
];
const store = normalizedStore({ entries, workspace: { root: "/tmp/root", key: "key" } } as WorkspaceMemoryStore, "/tmp/root", "key");
const candidates = retentionCandidatesForDiag(store, new Date("2026-01-02T00:00:00.000Z").getTime());
assert.equal(candidates.rendered.length, LONG_TERM_LIMITS.maxEntries);
assert.equal(candidates.typeCapped.length, 1);
assert.equal(candidates.globalCapped.length, 6);
assert.equal(candidates.typeCapped[0].entry.type, "feedback");
});
test("buildMemoryDiagJSON redacts previews, includes pending entries, and preserves summary fields", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-workspace-model-"));
try {
const active = entry("mem-active", "Remember password: sushi and file /Users/alice/private.txt", "decision");
const pending = { ...entry("mem-pending", "Pending api_key=secret-value", "project"), promotionAttempts: 1 };
await writeWorkspaceStore(root, [active]);
await writePendingJournal(root, [pending]);
const diag = await buildMemoryDiagJSON(root);
assert.equal(diag.version, 1);
assert.equal(diag.summary.storedActive, 1);
assert.equal(diag.summary.rendered, 1);
assert.equal(diag.summary.pending, 1);
assert.equal(diag.summary.rejectedLast7Days, 0);
assert.equal(diag.summary.corruptStoresQuarantinedLast30Days, 0);
assert.equal(diag.memories.length, 2);
assert.equal(diag.memories.find(memory => memory.id === "mem-pending")?.status, "pending_retry");
assert.ok(diag.memories.some(memory => memory.textPreview?.includes("[REDACTED]")));
assert.ok(diag.memories.some(memory => memory.textPreview?.includes("<path>")));
assert.ok(!JSON.stringify(diag).includes("sushi"));
assert.ok(!JSON.stringify(diag).includes("secret-value"));
assert.equal(diag.workspace.key, await workspaceKey(root));
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memoryDiagJSONFromSnapshot serializes an existing snapshot with fixed generatedAt", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-snapshot-json-"));
try {
const active = entry("mem-active", "Stable decision memory", "decision");
const pending = { ...entry("mem-pending", "Pending project memory", "project"), promotionAttempts: 1 };
await writeWorkspaceStore(root, [active]);
await writePendingJournal(root, [pending]);
const snapshot = await snapshotForOptions({ raw: false, workspace: root });
const generatedAt = "2026-05-02T00:00:00.000Z";
const diag = memoryDiagJSONFromSnapshot(root, snapshot, generatedAt);
assert.equal(diag.version, 1);
assert.equal(diag.generatedAt, generatedAt);
assert.equal(diag.memories, snapshot.memories);
assert.equal(diag.recentEvents, snapshot.recentEvents);
assert.equal(diag.summary, snapshot.summary);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("grouped evidence summaries match per-memory summaries for stored pending and absorbed memories", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-evidence-equivalence-"));
try {
const active = entry("mem-active", "Stable decision memory", "decision");
const pending = { ...entry("mem-pending", "Pending project memory", "project"), promotionAttempts: 1 };
await writeWorkspaceStore(root, [active]);
await writePendingJournal(root, [pending]);
await appendEvidenceEvents(root, [
evidence({ memory: { memoryId: "mem-active", type: "decision", source: "compaction", status: "active" }, reasonCodes: ["stored_reason"] }),
evidence({ type: "pending_memory_appended", phase: "pending_journal", outcome: "accepted", memory: { memoryId: "mem-pending", type: "project", source: "compaction" }, reasonCodes: ["pending_reason"] }),
evidence({ type: "promotion_absorbed_exact", phase: "promotion", outcome: "absorbed", memory: { memoryId: "mem-absorbed", type: "feedback", source: "compaction" }, reasonCodes: ["same_exact_key"] }),
]);
const grouped = groupEvidenceByMemoryId(await queryEvidenceEvents(root));
for (const id of ["mem-active", "mem-pending", "mem-absorbed"]) {
const oldSummary = await summarizeMemoryEvidence(root, { memoryId: id });
const groupedSummary = groupedEvidenceSummary(grouped, id);
assert.deepEqual(groupedSummary.eventIds, oldSummary.eventIds);
assert.deepEqual(groupedSummary.reasonCodes, oldSummary.reasonCodes);
}
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("buildMemoryDiagJSON preserves evidence ids and reason codes for stored pending and absorbed memories", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-evidence-rows-"));
try {
const active = entry("mem-active", "Stable decision memory", "decision");
const pending = { ...entry("mem-pending", "Pending project memory", "project"), promotionAttempts: 1 };
await writeWorkspaceStore(root, [active]);
await writePendingJournal(root, [pending]);
const events = await appendEvidenceEvents(root, [
evidence({ memory: { memoryId: "mem-active", type: "decision", source: "compaction", status: "active" }, reasonCodes: ["stored_reason"] }),
evidence({ type: "pending_memory_appended", phase: "pending_journal", outcome: "accepted", memory: { memoryId: "mem-pending", type: "project", source: "compaction" }, reasonCodes: ["pending_reason"] }),
evidence({ type: "promotion_absorbed_exact", phase: "promotion", outcome: "absorbed", memory: { memoryId: "mem-absorbed", type: "feedback", source: "compaction" }, reasonCodes: ["same_exact_key"] }),
]);
const diag = await buildMemoryDiagJSON(root);
const activeRow = diag.memories.find(memory => memory.id === "mem-active");
const pendingRow = diag.memories.find(memory => memory.id === "mem-pending");
const absorbedRow = diag.memories.find(memory => memory.id === "mem-absorbed");
assert.ok(activeRow);
assert.ok(pendingRow);
assert.ok(absorbedRow);
assert.deepEqual(activeRow.evidenceEventIds, [events[0].eventId]);
assert.ok(activeRow.reasonCodes.includes("stored_reason"));
assert.deepEqual(pendingRow.evidenceEventIds, [events[1].eventId]);
assert.ok(pendingRow.reasonCodes.includes("pending_reason"));
assert.deepEqual(absorbedRow.evidenceEventIds, [events[2].eventId]);
assert.ok(absorbedRow.reasonCodes.includes("same_exact_key"));
assert.ok(absorbedRow.reasonCodes.includes("absorbed_duplicate"));
} finally {
await rm(root, { recursive: true, force: true });
}
});
-779
View File
@@ -1,779 +0,0 @@
import test from "node:test";
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { mkdtempSync } from "node:fs";
import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { appendEvidenceEvents, type EvidenceEventInput } from "../src/evidence-log.ts";
import type { LongTermMemoryEntry, WorkspaceMemoryStore } from "../src/types.ts";
import { LONG_TERM_LIMITS, PROMOTION_RETRY_LIMITS, type PendingMemoryJournalStore } from "../src/types.ts";
import { extractionRejectionLogPath, workspaceEvidenceLogPath, workspaceKey, workspaceMemoryPath, workspacePendingJournalPath } from "../src/paths.ts";
const execFileAsync = promisify(execFile);
const repoRoot = join(dirname(fileURLToPath(import.meta.url)), "..");
function entry(id: string, text: string, type: LongTermMemoryEntry["type"]): LongTermMemoryEntry {
const now = new Date().toISOString();
return {
id,
type,
text,
source: "compaction",
confidence: 0.75,
status: "active",
createdAt: now,
updatedAt: now,
};
}
async function writeWorkspaceStore(root: string, entries: LongTermMemoryEntry[], options: { lastActivityAt?: string; omitLastActivityAt?: boolean } = {}): Promise<void> {
const key = await workspaceKey(root);
const path = await workspaceMemoryPath(root);
const now = new Date().toISOString();
const store: WorkspaceMemoryStore = {
version: 1,
workspace: { root, key },
limits: { maxRenderedChars: LONG_TERM_LIMITS.maxRenderedChars, maxEntries: LONG_TERM_LIMITS.maxEntries },
entries,
migrations: [],
updatedAt: now,
};
if (!options.omitLastActivityAt) store.lastActivityAt = options.lastActivityAt ?? now;
await mkdir(dirname(path), { recursive: true });
await writeFile(path, JSON.stringify(store, null, 2), "utf8");
}
async function runMemoryDiagStatusVerbose(root: string): Promise<string> {
return runMemoryDiag(["status", "--workspace", root, "--verbose"]);
}
async function runMemoryDiag(args: string[]): Promise<string> {
const { stdout } = await runMemoryDiagResult(args);
return stdout;
}
async function runMemoryDiagResult(args: string[], options: { cwd?: string } = {}): Promise<{ stdout: string; stderr: string }> {
const { stdout, stderr } = await execFileAsync(process.execPath, [
"--experimental-strip-types",
"scripts/memory-diag.ts",
...args,
], { cwd: options.cwd ?? repoRoot });
return { stdout: stdout.trim(), stderr: stderr.trim() };
}
async function writePendingJournal(root: string, entries: LongTermMemoryEntry[]): Promise<void> {
const key = await workspaceKey(root);
const path = await workspacePendingJournalPath(root);
const store: PendingMemoryJournalStore = {
version: 1,
workspace: { root, key },
entries,
updatedAt: new Date().toISOString(),
};
await mkdir(dirname(path), { recursive: true });
await writeFile(path, JSON.stringify(store, null, 2), "utf8");
}
async function writeRejectionRecords(records: unknown[]): Promise<void> {
const path = extractionRejectionLogPath();
await mkdir(dirname(path), { recursive: true });
await writeFile(path, records.map(record => JSON.stringify(record)).join("\n") + "\n", "utf8");
}
function evidence(overrides: Partial<EvidenceEventInput>): EvidenceEventInput {
return {
type: "promotion_promoted",
phase: "promotion",
outcome: "promoted",
memory: { memoryId: "mem-rendered", type: "feedback", source: "explicit", status: "active" },
reasonCodes: ["new_workspace_entry"],
...overrides,
};
}
test("status handles missing workspace store as empty", async () => {
const root = mkdtempSync(join(tmpdir(), "opencode-memory-diag-missing-health-"));
try {
const stdout = await runMemoryDiag(["status", "--workspace", root]);
assert.match(stdout, /Memory status/);
assert.match(stdout, /active memories: 0/);
assert.match(stdout, /pending: 0/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("status verbose handles missing workspace store as empty", async () => {
const root = mkdtempSync(join(tmpdir(), "opencode-memory-diag-missing-quality-"));
try {
const stdout = await runMemoryDiag(["status", "--workspace", root, "--verbose"]);
assert.match(stdout, /Memory status inspection/);
assert.match(stdout, /Caps:/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("coverage and disappearances handle missing workspace store as empty", async () => {
const root = mkdtempSync(join(tmpdir(), "opencode-memory-diag-missing-inspection-"));
try {
const coverageStdout = await runMemoryDiag(["coverage", "--workspace", root]);
assert.match(coverageStdout, /no_evidence: 0/);
assert.match(coverageStdout, /Per-memory rows:\n\s+\(none\)/);
const missingStdout = await runMemoryDiag(["missing", "--workspace", root]);
assert.match(missingStdout, /No missing memories found\./);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("status with unsupported all flag shows usage error", async () => {
const root = mkdtempSync(join(tmpdir(), "opencode-memory-diag-conflicting-flags-"));
try {
await assert.rejects(
runMemoryDiagResult(["status", "--all", "--workspace", root]),
(error: unknown) => {
const err = error as { code?: number; stderr?: string };
assert.notEqual(err.code, 0);
assert.match(err.stderr ?? "", /status does not accept --all/);
assert.match(err.stderr ?? "", /Usage:/);
return true;
},
);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory-diag reports unexpected command errors without stack traces", async () => {
const root = mkdtempSync(join(tmpdir(), "opencode-memory-diag-error-boundary-"));
try {
await writeWorkspaceStore(root, []);
// Invalid rejection filter values are parse-caught before dispatch, so this
// fixture exercises the same top-level boundary with a command-thrown Error.
await mkdir(await workspaceEvidenceLogPath(root), { recursive: true });
await assert.rejects(
runMemoryDiagResult(["status", "--workspace", root]),
(error: unknown) => {
const err = error as { code?: number; stderr?: string };
assert.notEqual(err.code, 0);
assert.match(err.stderr ?? "", /memory-diag failed:/);
assert.doesNotMatch(err.stderr ?? "", /\n\s+at\s|Error:/);
return true;
},
);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory-diag defaults to status when no subcommand is supplied", async () => {
const result = await runMemoryDiagResult([]);
assert.match(result.stdout, /Memory status/);
assert.match(result.stdout, /Key metrics:/);
assert.equal(result.stderr, "");
});
test("removed legacy aliases return unknown subcommand", async () => {
const root = mkdtempSync(join(tmpdir(), "opencode-memory-diag-legacy-health-"));
try {
for (const command of ["health", "quality", "rejections", "disappearances", "trace"]) {
await assert.rejects(
runMemoryDiagResult([command, "--workspace", root]),
(error: unknown) => {
const err = error as { code?: number; stderr?: string };
assert.notEqual(err.code, 0);
assert.match(err.stderr ?? "", new RegExp(`Unknown subcommand: ${command}`));
assert.match(err.stderr ?? "", /Usage:/);
return true;
},
);
}
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory health reports stored vs rendered retention counts", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-"));
try {
const entries: LongTermMemoryEntry[] = [
...Array.from({ length: 17 }, (_, i) => entry(`feedback-${i}`, `Unique feedback preference for memory health ${i}`, "feedback")),
...Array.from({ length: 11 }, (_, i) => entry(`decision-${i}`, `Unique durable decision for memory health ${i}`, "decision")),
];
await writeWorkspaceStore(root, entries);
const stdout = await runMemoryDiagStatusVerbose(root);
assert.match(stdout, /Memory status inspection/);
assert.match(stdout, /active: 28 \/ 28/);
assert.match(stdout, /rendered: 20/);
assert.match(stdout, /feedback: 17 \/ 10 FULL/);
assert.match(stdout, /Top rendered candidates:\n\s+- strength=/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory health reports dormancy and retention monitoring deprecations", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-"));
try {
const lastActivityAt = new Date(Date.now() - 19 * 24 * 60 * 60 * 1000).toISOString();
const entries = Array.from({ length: 10 }, (_, i) => ({
...entry(`monitoring-${i}`, `Unique monitoring memory ${i} for retention health`, i % 2 === 0 ? "feedback" : "decision"),
userImportance: i < 4 ? "high" as const : "normal" as const,
safetyCritical: i < 6,
reinforcementCount: i < 2 ? 6 : 0,
}));
await writeWorkspaceStore(root, entries, { lastActivityAt });
const stdout = await runMemoryDiagStatusVerbose(root);
assert.match(stdout, /Dormancy:/);
assert.match(stdout, /wall days since activity: 19\.0/);
assert.match(stdout, /dormant discount active: yes/);
assert.match(stdout, /dormant days past grace: 5\.0/);
assert.match(stdout, /Top rendered candidates:/);
assert.match(stdout, /Weakest active memories:/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory health reports global cap overflow separately from type caps", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-"));
try {
const entries: LongTermMemoryEntry[] = [
...Array.from({ length: 10 }, (_, i) => entry(`global-feedback-${i}`, `Unique global feedback preference ${i}`, "feedback")),
...Array.from({ length: 10 }, (_, i) => entry(`global-decision-${i}`, `Unique global durable decision ${i}`, "decision")),
...Array.from({ length: 8 }, (_, i) => entry(`global-project-${i}`, `Unique global project fact ${i}`, "project")),
...Array.from({ length: 6 }, (_, i) => entry(`global-reference-${i}`, `Unique global reference fact ${i}`, "reference")),
];
await writeWorkspaceStore(root, entries);
const stdout = await runMemoryDiagStatusVerbose(root);
assert.match(stdout, /rendered: 28/);
assert.match(stdout, /type-capped entries: 0/);
assert.match(stdout, /global-cap overflow: 6/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory health reports missing dormancy and non-alert monitoring defaults", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-"));
try {
await writeWorkspaceStore(root, [], { omitLastActivityAt: true });
const stdout = await runMemoryDiagStatusVerbose(root);
assert.match(stdout, /lastActivityAt: \(missing\)/);
assert.match(stdout, /wall days since activity: unknown/);
assert.match(stdout, /dormant discount active: no/);
assert.match(stdout, /Top rendered candidates:\n\s+\(none\)/);
assert.match(stdout, /Weakest active memories:\n\s+\(none\)/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory health --json prints parseable privacy-safe diagnostics matching human counts", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-json-"));
try {
const rendered = { ...entry("mem-rendered", "Prefer small focused changes", "feedback"), source: "explicit" as const };
const secret = { ...entry("mem-secret", "Use password: sushi only in test fixtures", "decision"), source: "manual" as const };
const superseded = { ...entry("mem-old", "Old decision that was superseded", "decision"), status: "superseded" as const };
const pending = { ...entry("mem-pending", "Retry this pending memory later", "project"), promotionAttempts: 1 };
await writeWorkspaceStore(root, [rendered, secret, superseded]);
await writePendingJournal(root, [pending]);
await appendEvidenceEvents(root, [
evidence({ type: "extraction_candidate_rejected", phase: "extraction", outcome: "rejected", memory: { memoryId: "mem-rejected", type: "feedback", source: "explicit" }, reasonCodes: ["raw_secret"], textPreview: "password: sushi should not leak" }),
evidence({ type: "storage_corrupt_json_quarantined", phase: "storage", outcome: "quarantined", memory: undefined, reasonCodes: ["invalid_json"] }),
evidence({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "mem-rendered", type: "feedback", source: "explicit", status: "active" }, reasonCodes: ["new_workspace_entry"] }),
evidence({ type: "render_selected", phase: "render", outcome: "rendered", memory: { memoryId: "mem-rendered", type: "feedback", source: "explicit", status: "active" }, reasonCodes: ["within_caps", "within_char_budget"] }),
]);
const human = await runMemoryDiagStatusVerbose(root);
const jsonText = await runMemoryDiag(["status", "--workspace", root, "--json"]);
const parsed = JSON.parse(jsonText) as {
version: 1;
summary: { storedActive: number; rendered: number; pending: number; rejectedLast7Days: number; corruptStoresQuarantinedLast30Days: number };
memories: Array<{ id: string; status: string; reasonCodes: string[]; evidenceEventIds: string[]; textPreview?: string }>;
recentEvents: Array<{ eventId: string; type: string; outcome: string; createdAt: string; reasonCodes: string[] }>;
};
assert.equal(parsed.version, 1);
assert.equal(parsed.summary.storedActive, Number(human.match(/active: (\d+) \/ 28/)?.[1]));
assert.equal(parsed.summary.rendered, Number(human.match(/rendered: (\d+)/)?.[1]));
assert.equal(parsed.summary.pending, 1);
assert.equal(parsed.summary.rejectedLast7Days, 1);
assert.equal(parsed.summary.corruptStoresQuarantinedLast30Days, 1);
assert.ok(parsed.recentEvents.some(event => event.eventId && event.type === "render_selected" && event.outcome === "rendered" && event.createdAt && event.reasonCodes.includes("within_caps")));
assert.ok(parsed.memories.find(memory => memory.id === "mem-rendered")?.evidenceEventIds.length);
assert.ok(!jsonText.includes("sushi"));
assert.ok(jsonText.trim().startsWith("{"));
assert.ok(jsonText.trim().endsWith("}"));
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory-diag explain shows rendered, omitted, pending, and evidence reason status", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-explain-"));
try {
const rendered = { ...entry("mem-rendered", "Rendered feedback wins the render set", "feedback"), source: "explicit" as const };
const superseded = { ...entry("mem-superseded", "Superseded memory is not rendered", "decision"), status: "superseded" as const };
const typeCapped = Array.from({ length: 11 }, (_, i) => entry(`mem-type-${i}`, `Type cap feedback memory ${i}`, "feedback"));
const globalCapped = [
...Array.from({ length: 10 }, (_, i) => entry(`mem-g-feedback-${i}`, `Global cap feedback memory ${i}`, "feedback")),
...Array.from({ length: 10 }, (_, i) => entry(`mem-g-decision-${i}`, `Global cap decision memory ${i}`, "decision")),
...Array.from({ length: 8 }, (_, i) => entry(`mem-g-project-${i}`, `Global cap project memory ${i}`, "project")),
...Array.from({ length: 7 }, (_, i) => entry(`mem-g-reference-${i}`, `Global cap reference memory ${i}`, "reference")),
];
const charBudget = { ...entry("mem-char-budget", "This active memory cannot fit the tiny character budget", "project") };
await writeWorkspaceStore(root, [rendered, superseded, ...typeCapped, ...globalCapped, charBudget]);
const key = await workspaceKey(root);
const path = await workspaceMemoryPath(root);
const raw = JSON.parse(await readFile(path, "utf8")) as WorkspaceMemoryStore;
raw.limits.maxRenderedChars = 100;
raw.workspace = { root, key };
await writeFile(path, JSON.stringify(raw, null, 2), "utf8");
const retry = { ...entry("mem-pending-retry", "Pending retry memory", "project"), promotionAttempts: 1, lastPromotionFailureReason: "capacity_rejected" };
const exhausted = { ...entry("mem-pending-capacity", "Pending capacity rejected memory", "project"), promotionAttempts: PROMOTION_RETRY_LIMITS.maxExplicitAttempts, lastPromotionFailureReason: "capacity_rejected" };
await writePendingJournal(root, [retry, exhausted]);
await appendEvidenceEvents(root, [
evidence({ type: "promotion_absorbed_exact", phase: "promotion", outcome: "absorbed", memory: { memoryId: "mem-absorbed", type: "feedback", source: "explicit" }, reasonCodes: ["same_exact_key"] }),
evidence({ type: "storage_corrupt_json_quarantined", phase: "storage", outcome: "quarantined", memory: undefined, reasonCodes: ["invalid_json"] }),
]);
const stdout = await runMemoryDiag(["explain", "--workspace", root]);
assert.match(stdout, /Memory mem-rendered: rendered/);
assert.match(stdout, /Memory mem-superseded: omitted_superseded/);
assert.match(stdout, /omitted_type_cap/);
assert.match(stdout, /omitted_global_cap/);
assert.match(stdout, /omitted_char_budget/);
assert.match(stdout, /Memory mem-pending-retry: pending_retry/);
assert.match(stdout, /Memory mem-pending-capacity: pending_rejected_capacity/);
assert.match(stdout, /Memory mem-absorbed: omitted_absorbed_duplicate/);
assert.match(stdout, /quarantined_corrupt_store/);
assert.match(stdout, /- strength=\d+\.\d{3}, type=feedback, source=explicit/);
assert.match(stdout, /- evidence: .*promotion_absorbed_exact/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory-diag explain --memory prints lifecycle relations and redacts secrets", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-trace-"));
try {
await writeWorkspaceStore(root, [
{ ...entry("mem-life", "Old token password: sushi should be redacted", "decision"), status: "superseded" as const },
entry("mem-new", "Replacement memory", "decision"),
]);
await appendEvidenceEvents(root, [
evidence({ type: "extraction_candidate_accepted", phase: "extraction", outcome: "accepted", memory: { memoryId: "mem-life", type: "decision", source: "compaction" }, reasonCodes: ["quality_gate_passed"], textPreview: "password: sushi" }),
evidence({ type: "pending_memory_appended", phase: "pending_journal", outcome: "accepted", memory: { memoryId: "mem-life", type: "decision", source: "compaction" }, relations: [{ role: "pending", memory: { memoryId: "mem-life" } }], reasonCodes: ["pending_journal_append"] }),
evidence({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "mem-life", type: "decision", source: "compaction" }, reasonCodes: ["new_workspace_entry"] }),
evidence({ type: "memory_reinforced", phase: "reinforcement", outcome: "reinforced", memory: { memoryId: "mem-life", type: "decision", source: "compaction" }, relations: [{ role: "reinforced_by", memory: { memoryId: "mem-duplicate" } }], reasonCodes: ["duplicate_exact"] }),
evidence({ type: "promotion_superseded", phase: "promotion", outcome: "superseded", memory: { memoryId: "mem-life", type: "decision", source: "compaction" }, relations: [{ role: "superseded_by", memory: { memoryId: "mem-new" } }], reasonCodes: ["superseded_existing"] }),
evidence({ type: "render_omitted", phase: "render", outcome: "omitted", memory: { memoryId: "mem-life", type: "decision", source: "compaction" }, relations: [{ role: "omitted", memory: { memoryId: "mem-life" } }], reasonCodes: ["superseded"] }),
]);
const stdout = await runMemoryDiag(["explain", "--workspace", root, "--memory", "mem-life"]);
assert.match(stdout, /Memory mem-life: omitted_superseded/);
assert.match(stdout, /Lifecycle:/);
assert.match(stdout, /extraction_candidate_accepted: accepted; reasons=quality_gate_passed/);
assert.match(stdout, /pending_memory_appended: accepted; reasons=pending_journal_append/);
assert.match(stdout, /promotion_superseded: superseded; reasons=superseded_existing; .*superseded_by=mem-new/);
assert.match(stdout, /memory_reinforced: reinforced; reasons=duplicate_exact; .*reinforced_by=mem-duplicate/);
assert.match(stdout, /Superseded by:\n- mem-new/);
assert.match(stdout, /Reinforced by:\n- mem-duplicate/);
assert.ok(!stdout.includes("sushi"));
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory-diag explain positional memory id prints lifecycle", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-explain-positional-"));
try {
await writeWorkspaceStore(root, [
{ ...entry("mem-life", "Old token password: sushi should be redacted", "decision"), status: "superseded" as const },
entry("mem-new", "Replacement memory", "decision"),
]);
await appendEvidenceEvents(root, [
evidence({ type: "extraction_candidate_accepted", phase: "extraction", outcome: "accepted", memory: { memoryId: "mem-life", type: "decision", source: "compaction" }, reasonCodes: ["quality_gate_passed"], textPreview: "password: sushi" }),
evidence({ type: "promotion_superseded", phase: "promotion", outcome: "superseded", memory: { memoryId: "mem-life", type: "decision", source: "compaction" }, relations: [{ role: "superseded_by", memory: { memoryId: "mem-new" } }], reasonCodes: ["superseded_existing"] }),
]);
const stdout = await runMemoryDiag(["explain", "mem-life", "--workspace", root]);
assert.match(stdout, /Memory mem-life: omitted_superseded/);
assert.match(stdout, /Lifecycle:/);
assert.match(stdout, /promotion_superseded: superseded; reasons=superseded_existing; .*superseded_by=mem-new/);
assert.match(stdout, /Superseded by:\n- mem-new/);
assert.ok(!stdout.includes("sushi"));
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("memory-diag explain --memory reports unknown IDs", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-trace-unknown-"));
try {
const stdout = await runMemoryDiag(["explain", "--workspace", root, "--memory", "missing-memory"]);
assert.match(stdout, /Memory missing-memory: unknown/);
assert.match(stdout, /Lifecycle:\n\(none\)/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
async function setupQualityFixture(): Promise<string> {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-quality-"));
const key = await workspaceKey(root);
const now = new Date().toISOString();
const entries = Array.from({ length: LONG_TERM_LIMITS.maxEntries }, (_, i) => ({
...entry(`quality-${i}`, `Quality fixture memory ${i}`, i % 2 === 0 ? "feedback" as const : "decision" as const),
retentionClock: i === 0 ? -1 : Date.now(),
}));
await writeWorkspaceStore(root, entries);
await appendEvidenceEvents(root, [
evidence({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "quality-1", type: "decision", source: "compaction" }, reasonCodes: ["new_workspace_entry"] }),
evidence({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "quality-historical", type: "reference", source: "compaction" }, reasonCodes: ["new_workspace_entry"] }),
]);
await writeRejectionRecords([
{ timestamp: now, workspaceKey: key, workspaceRootHash: "hash", type: "decision", source: "compaction", text: "Implemented phase 2 and updated tests", reasons: ["bad_decision"] },
{ timestamp: now, type: "feedback", source: "compaction", text: "Wave 1 completed successfully", reasons: ["progress_snapshot", "bad_feedback"] },
]);
return root;
}
test("status verbose output includes summary and aggregate inspection counts", async () => {
const root = await setupQualityFixture();
try {
const stdout = await runMemoryDiag(["status", "--workspace", root, "--verbose"]);
assert.match(stdout, /Memory status inspection/);
assert.match(stdout, /Summary: Workspace memory quality is degraded:/);
assert.match(stdout, /Caps:\n\s+active: 28 \/ 28/);
assert.match(stdout, /decision: 14 \/ 10 FULL/);
assert.match(stdout, /feedback: 14 \/ 10 FULL/);
assert.match(stdout, /Retention clocks:\n\s+present: 27\n\s+missing: 0\n\s+invalid: 1/);
assert.match(stdout, /Evidence:\n\s+current with evidence: 1\n\s+current without evidence: 27/);
assert.match(stdout, /Rejection scoping:\n\s+total records: 2\n\s+workspace scoped: 1\n\s+legacy unscoped: 1/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("status --json includes summary status fields", async () => {
const root = await setupQualityFixture();
try {
const stdout = await runMemoryDiag(["status", "--workspace", root, "--json"]);
const parsed = JSON.parse(stdout) as {
summary: { storedActive: number; status: string; evidenceCoveragePercent: number; needsAttention: string[] };
};
assert.equal(parsed.summary.storedActive, LONG_TERM_LIMITS.maxEntries);
assert.equal(parsed.summary.status, "degraded");
assert.equal(typeof parsed.summary.evidenceCoveragePercent, "number");
assert.ok(parsed.summary.needsAttention.length > 0);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("status default output is concise and actionable", async () => {
const root = await setupQualityFixture();
try {
const stdout = await runMemoryDiag(["status", "--workspace", root]);
assert.match(stdout, /OK Memory status|WARNING Memory status|DEGRADED Memory status/);
assert.match(stdout, /Key metrics:/);
assert.match(stdout, /active memories:/);
assert.match(stdout, /rendered:/);
assert.match(stdout, /pending:/);
assert.match(stdout, /rejected 7d:/);
assert.match(stdout, /evidence coverage:/);
assert.match(stdout, /Needs attention:/);
assert.match(stdout, /Suggested next steps:/);
assert.doesNotMatch(stdout, /Caps:|Retention clocks:|Rejection scoping:|Dormancy:/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("status --verbose includes detailed diagnostics", async () => {
const root = await setupQualityFixture();
try {
const stdout = await runMemoryDiag(["status", "--workspace", root, "--verbose"]);
assert.match(stdout, /Memory status inspection/);
assert.match(stdout, /Caps:/);
assert.match(stdout, /Retention clocks:/);
assert.match(stdout, /Evidence:/);
assert.match(stdout, /Rejection scoping:/);
assert.match(stdout, /Dormancy:/);
assert.match(stdout, /Top rendered candidates:/);
assert.match(stdout, /Weakest active memories:/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("status --json includes additive summary fields", async () => {
const root = await setupQualityFixture();
try {
const stdout = await runMemoryDiag(["status", "--workspace", root, "--json"]);
const parsed = JSON.parse(stdout) as {
version: 1;
summary: {
storedActive: number;
rendered: number;
pending: number;
rejectedLast7Days: number;
corruptStoresQuarantinedLast30Days: number;
status?: string;
evidenceCoveragePercent?: number;
needsAttention?: string[];
suggestedNextSteps?: string[];
};
memories: unknown[];
recentEvents: unknown[];
};
assert.equal(parsed.version, 1);
assert.equal(typeof parsed.summary.storedActive, "number");
assert.equal(typeof parsed.summary.rendered, "number");
assert.equal(typeof parsed.summary.pending, "number");
assert.equal(typeof parsed.summary.rejectedLast7Days, "number");
assert.equal(typeof parsed.summary.corruptStoresQuarantinedLast30Days, "number");
assert.match(parsed.summary.status ?? "", /ok|warning|degraded/);
assert.equal(typeof parsed.summary.evidenceCoveragePercent, "number");
assert.ok(Array.isArray(parsed.summary.needsAttention));
assert.ok(Array.isArray(parsed.summary.suggestedNextSteps));
assert.ok(Array.isArray(parsed.memories));
assert.ok(Array.isArray(parsed.recentEvents));
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("status non-tty and --no-emoji use text labels", async () => {
const root = await setupQualityFixture();
try {
const stdout = await runMemoryDiag(["status", "--workspace", root, "--no-emoji"]);
assert.match(stdout, /^DEGRADED Memory status/);
assert.doesNotMatch(stdout, /🧠|⚠️|✖️/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("coverage human output includes class counts and per-memory rows", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-coverage-"));
try {
await writeWorkspaceStore(root, [
entry("mem-full", "Full lifecycle memory", "feedback"),
entry("mem-render-only", "Render only memory", "decision"),
entry("mem-no-evidence", "No evidence memory", "project"),
]);
await appendEvidenceEvents(root, [
evidence({ type: "extraction_candidate_accepted", phase: "extraction", outcome: "accepted", memory: { memoryId: "mem-full", type: "feedback", source: "compaction" }, reasonCodes: ["quality_gate_passed"] }),
evidence({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "mem-full", type: "feedback", source: "compaction" }, reasonCodes: ["new_workspace_entry"] }),
evidence({ type: "render_selected", phase: "render", outcome: "rendered", memory: { memoryId: "mem-render-only", type: "decision", source: "compaction" }, reasonCodes: ["within_caps"] }),
]);
const stdout = await runMemoryDiag(["coverage", "--workspace", root]);
assert.match(stdout, /Class counts:/);
assert.match(stdout, /full_lifecycle: 1/);
assert.match(stdout, /render_only: 1/);
assert.match(stdout, /no_evidence: 1/);
assert.match(stdout, /Per-memory rows:/);
assert.match(stdout, /mem-full full_lifecycle .*extraction=1 promotion=1/);
assert.match(stdout, /mem-render-only render_only .*render=1/);
assert.match(stdout, /mem-no-evidence no_evidence .*total=0/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("coverage --json includes event counts", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-coverage-json-"));
try {
await writeWorkspaceStore(root, [entry("mem-full", "Full lifecycle memory", "feedback")]);
await appendEvidenceEvents(root, [
evidence({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "mem-full", type: "feedback", source: "compaction" }, reasonCodes: ["new_workspace_entry"] }),
]);
const stdout = await runMemoryDiag(["coverage", "--workspace", root, "--json"]);
const parsed = JSON.parse(stdout) as { memories: Array<{ id: string; eventCounts: { total: number; byType: Record<string, number> } }> };
const row = parsed.memories.find(memory => memory.id === "mem-full");
assert.equal(row?.eventCounts.total, 1);
assert.equal(row?.eventCounts.byType.promotion_promoted, 1);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("missing default output summarizes unknown disappearances", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-disappear-"));
try {
await writeWorkspaceStore(root, [entry("current", "Current memory", "feedback")]);
await appendEvidenceEvents(root, [
evidence({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "historical-unknown", type: "decision", source: "compaction" }, reasonCodes: ["new_workspace_entry"] }),
]);
const stdout = await runMemoryDiag(["missing", "--workspace", root]);
assert.match(stdout, /Missing memory summary/);
assert.match(stdout, /Total missing: 1/);
assert.match(stdout, /Explained: 0/);
assert.match(stdout, /Needs review: 1/);
assert.match(stdout, /Unknown disappearance samples:/);
assert.match(stdout, /- historical-unknown terminal=unknown reasons=none/);
assert.doesNotMatch(stdout, /Memory historical-unknown:/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("missing --verbose shows capacity details and render omitted type-cap observations", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-disappear-explain-"));
try {
await writeWorkspaceStore(root, [entry("current", "Current memory", "feedback")]);
await appendEvidenceEvents(root, [
evidence({
type: "memory_removed_capacity",
phase: "storage",
outcome: "removed",
memory: { memoryId: "capacity-loser", type: "decision", source: "compaction", status: "active" },
relations: [{ role: "removed", memory: { memoryId: "capacity-loser", type: "decision", source: "compaction", status: "active" } }],
reasonCodes: ["type_cap"],
details: { type: "decision", globalCap: 28, typeCap: 10, source: "compaction" },
}),
evidence({ type: "render_omitted", phase: "render", outcome: "omitted", memory: { memoryId: "render-loser", type: "feedback", source: "compaction" }, reasonCodes: ["type_cap"] }),
]);
const stdout = await runMemoryDiag(["missing", "--workspace", root, "--verbose"]);
assert.match(stdout, /Missing memory summary/);
assert.match(stdout, /capacity-loser: historical_absent_with_reason terminal=memory_removed_capacity reasons=type_cap/);
assert.match(stdout, /events:/);
assert.match(stdout, /memory_removed_capacity details: .*globalCap=28 .*typeCap=10/);
assert.match(stdout, /render-loser: historical_absent_with_reason terminal=render_omitted reasons=type_cap/);
assert.match(stdout, /render_omitted type-cap observation: reasons=type_cap/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("missing --json includes disappearances and additive summary", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-missing-json-"));
try {
await writeWorkspaceStore(root, [entry("current", "Current memory", "feedback")]);
await appendEvidenceEvents(root, [
evidence({ type: "promotion_promoted", phase: "promotion", outcome: "promoted", memory: { memoryId: "historical-unknown", type: "decision", source: "compaction" }, reasonCodes: ["new_workspace_entry"] }),
]);
const stdout = await runMemoryDiag(["missing", "--workspace", root, "--json"]);
const parsed = JSON.parse(stdout) as { disappearances: Array<{ id: string }>; summary: { total: number; explained: number; needsReview: number } };
assert.equal(parsed.summary.total, 1);
assert.equal(parsed.summary.explained, 0);
assert.equal(parsed.summary.needsReview, 1);
assert.equal(parsed.disappearances[0]?.id, "historical-unknown");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("rejected default output is concise with top reasons and samples", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-rejected-"));
try {
const key = await workspaceKey(root);
const now = new Date().toISOString();
await writeRejectionRecords([
{ timestamp: now, workspaceKey: key, type: "decision", source: "compaction", text: "Keep memory system boundary stable", reasons: ["bad_decision"] },
{ timestamp: now, workspaceKey: key, type: "feedback", source: "explicit", text: "Remember password: sushi from rejected sample", reasons: ["raw_secret", "bad_feedback"] },
{ timestamp: "2020-01-01T00:00:00.000Z", workspaceKey: key, type: "project", source: "manual", text: "Old rejected sample", reasons: ["bad_project"] },
]);
const stdout = await runMemoryDiag(["rejected", "--workspace", root]);
assert.match(stdout, /Rejected memory summary/);
assert.match(stdout, /Total rejected: 3/);
assert.match(stdout, /Unique texts: 3/);
assert.match(stdout, /Top reasons:/);
assert.match(stdout, /bad_decision\s+1/);
assert.match(stdout, /False-positive risk: low/);
assert.match(stdout, /Recent samples:/);
assert.match(stdout, /\[decision\] Keep memory system boundary stable/);
assert.ok(!stdout.includes("sushi"));
assert.doesNotMatch(stdout, /By origin:|Reason distribution \(raw records\):|Possible false-positive groups/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("rejected --verbose includes detailed distributions", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-rejected-verbose-"));
try {
const key = await workspaceKey(root);
const now = new Date().toISOString();
await writeRejectionRecords([
{ timestamp: now, workspaceKey: key, type: "decision", source: "compaction", text: "Memory system schema boundary should remain stable", reasons: ["bad_decision"] },
{ timestamp: now, workspaceKey: key, type: "feedback", source: "explicit", text: "Status update completed", reasons: ["bad_feedback"] },
]);
const stdout = await runMemoryDiag(["rejected", "--workspace", root, "--verbose"]);
assert.match(stdout, /Rejected memory summary/);
assert.match(stdout, /Reason distribution \(raw records\):/);
assert.match(stdout, /Reason distribution \(unique text\):/);
assert.match(stdout, /By origin:/);
assert.match(stdout, /Possible false-positive groups \(heuristic, not deterministic\):/);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("rejected --json includes quality summary and false-positive risk", async () => {
const root = await mkdtemp(join(tmpdir(), "opencode-memory-diag-rejected-json-"));
try {
const key = await workspaceKey(root);
const now = new Date().toISOString();
await writeRejectionRecords([
{ timestamp: now, workspaceKey: key, type: "decision", source: "compaction", text: "Memory system schema boundary should remain stable", reasons: ["bad_decision"] },
{ timestamp: now, workspaceKey: key, type: "decision", source: "compaction", text: "Memory system schema boundary should remain stable", reasons: ["bad_decision"] },
]);
const stdout = await runMemoryDiag(["rejected", "--workspace", root, "--json"]);
const parsed = JSON.parse(stdout) as { totalRecords: number; uniqueTexts: number; falsePositiveRisk: string; possibleFalsePositiveGroups: Record<string, { count: number }> };
assert.equal(parsed.totalRecords, 2);
assert.equal(parsed.uniqueTexts, 1);
assert.equal(parsed.falsePositiveRisk, "high");
assert.equal(parsed.possibleFalsePositiveGroups.architecture_like_possible_false_positive.count, 1);
} finally {
await rm(root, { recursive: true, force: true });
}
});
-27
View File
@@ -136,33 +136,6 @@ test("decision must be future-facing rule, not completed implementation note", (
assert.equal(assessMemoryQuality({ type: "decision", text: "Added semantic merge tests in the previous wave", source: "compaction" }).accepted, false);
});
test("bad_decision 3-tier gate: architecture-like decisions accepted without future-rule imperative", () => {
const architectureLikeCases = [
{ text: "Rule 不在記憶系統 schema 內,歸用戶(agent.md / claude.md),系統最多到 Preference + Suggestion", type: "decision" as const },
{ text: "Ghost memory root cause: normalization 把 capacity losers 從 store 移除時沒有 emit terminal evidence", type: "decision" as const },
{ text: "BASE_HALF_LIFE_DAYS 應從 60 降低到 45", type: "decision" as const },
{ text: "採用 decay-rate 模型取代 priority+penalty 模型", type: "decision" as const },
{ text: "從 scoring 移除 confidence,目前是固定值無意義", type: "decision" as const },
];
const stillRejectedCases = [
{ text: "Implemented phase 2 and updated tests", type: "decision" as const },
{ text: "Implemented CI_SCHEMA_UPDATE for compatibility run 42", type: "decision" as const },
{ text: "Session reviewed the architecture model changes", type: "decision" as const },
{ text: "Some random text with no architecture keywords or future rules", type: "decision" as const },
];
for (const entry of architectureLikeCases) {
const result = assessMemoryQuality({ ...entry, source: "compaction" });
assert.equal(result.reasons.includes("bad_decision"), false, `${entry.text} -> ${result.reasons.join(",")}`);
}
for (const entry of stillRejectedCases) {
const result = assessMemoryQuality({ ...entry, source: "compaction" });
assert.equal(result.reasons.includes("bad_decision"), true, `${entry.text} -> ${result.reasons.join(",")}`);
}
});
test("shared quality gate owns extractor low-quality syntax rejections", () => {
const rejected = [
{ type: "project" as const, text: "fix: add new feature" },
+10 -368
View File
@@ -1,6 +1,6 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { mkdir, mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { MemoryV2Plugin } from "../src/plugin.ts";
@@ -8,10 +8,9 @@ import { loadSessionState, saveSessionState } from "../src/session-state.ts";
import { parseWorkspaceMemoryCandidates } from "../src/extractors.ts";
import type { OpenError } from "../src/types.ts";
import { PROMOTION_RETRY_LIMITS, WORKSPACE_MEMORY_CACHE_LIMITS } from "../src/types.ts";
import { sessionStatePath, workspaceMemoryPath, workspacePendingJournalPath } from "../src/paths.ts";
import { workspaceMemoryPath, workspacePendingJournalPath } from "../src/paths.ts";
import { loadPendingJournal, savePendingJournal, memoryKey } from "../src/pending-journal.ts";
import { loadWorkspaceMemory, updateWorkspaceMemory } from "../src/workspace-memory.ts";
import { queryEvidenceEvents } from "../src/evidence-log.ts";
// Mock client for root session (not a sub-agent)
function mockRootClient() {
@@ -339,69 +338,6 @@ test("compaction prompt forbids progress and session-internal memory candidates"
}
});
test("compaction prompt includes existing-memory wording reuse guidance", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-prompt-"));
try {
const plugin = await MemoryV2Plugin({ directory: tmpDir, client: mockRootClient() });
const output = { prompt: "", context: [] as string[] };
await (plugin as Record<string, Function>)["experimental.session.compacting"](
{ sessionID: "prompt-wording-reuse-session", model: {} },
output,
);
assert.match(output.prompt, /Existing workspace memory may already contain durable facts/);
assert.match(output.prompt, /do not create a rephrased duplicate/);
assert.match(output.prompt, /reuse the existing memory wording exactly whenever possible/);
assert.match(output.prompt, /new, materially corrected, or materially more specific/);
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
});
test("compaction prompt does not introduce CRUD memory directives", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-prompt-"));
try {
const plugin = await MemoryV2Plugin({ directory: tmpDir, client: mockRootClient() });
const output = { prompt: "", context: [] as string[] };
await (plugin as Record<string, Function>)["experimental.session.compacting"](
{ sessionID: "prompt-no-crud-session", model: {} },
output,
);
assert.doesNotMatch(output.prompt, /\bREPLACE\b/);
assert.doesNotMatch(output.prompt, /\bDROP\b/);
assert.doesNotMatch(output.prompt, /\bREINFORCE\s+\[M/);
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
});
test("compaction prompt preserves Memory candidates output format", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-prompt-"));
try {
const plugin = await MemoryV2Plugin({ directory: tmpDir, client: mockRootClient() });
const output = { prompt: "", context: [] as string[] };
await (plugin as Record<string, Function>)["experimental.session.compacting"](
{ sessionID: "prompt-format-session", model: {} },
output,
);
assert.match(
output.prompt,
/Format when there ARE durable memories:\nMemory candidates:\n- \[feedback\|decision\|project\|reference\] future-facing durable fact/,
);
assert.match(
output.prompt,
/Format when there are NO durable memories:\nMemory candidates:\n\(none\)/,
);
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
});
test("compaction hook merges existing output.context from other plugins", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-test-"));
@@ -514,62 +450,6 @@ test("chat system transform reuses frozen rendered workspace snapshot", async ()
}
});
test("chat system transform degrades gracefully when workspace memory JSON is corrupt", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-test-"));
try {
const path = await workspaceMemoryPath(tmpDir);
await mkdir(dirname(path), { recursive: true });
await writeFile(path, "{ invalid json", "utf8");
const plugin = await MemoryV2Plugin({
directory: tmpDir,
client: mockRootClient(),
});
const output = { system: ["base header"] };
await assert.doesNotReject(async () => {
await (plugin as Record<string, Function>)["experimental.chat.system.transform"](
{ sessionID: "corrupt-json-session", model: {} },
output,
);
});
assert.deepEqual(output.system, ["base header"]);
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
});
test("hook failure emits hook_failed evidence without raw tool output", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-test-"));
try {
const statePath = await sessionStatePath(tmpDir, "hook-failure-session");
await mkdir(dirname(statePath), { recursive: true });
await mkdir(statePath, { recursive: true });
const plugin = await MemoryV2Plugin({ directory: tmpDir, client: mockRootClient() });
await (plugin as Record<string, Function>)["tool.execute.after"](
{
tool: "bash",
sessionID: "hook-failure-session",
args: { command: "cat secret-output.txt" },
},
{ output: "raw tool output password: sushi should not be in evidence", exitCode: 1 },
);
const events = await queryEvidenceEvents(tmpDir, { types: ["hook_failed"] });
const raw = JSON.stringify(events);
assert.equal(events.length, 1);
assert.equal(events[0].reasonCodes.includes("tool.execute.after"), true);
assert.equal(raw.includes("raw tool output"), false);
assert.equal(raw.includes("sushi"), false);
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
});
test("no compaction: owned explicit memory is not promoted by unrelated next session start", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-test-"));
@@ -892,69 +772,6 @@ test("session.compacted promotes pending memories to workspace memory and clears
const workspacePrompt = after.system.find((part: string) => part.startsWith("Workspace memory"));
assert.match(workspacePrompt ?? "", /Use frozen rendered snapshots for cache stability/);
const workspace = await loadWorkspaceMemory(tmpDir);
const promoted = workspace.entries.find(entry => entry.id === "mem_pending_1");
assert.ok(typeof promoted?.retentionClock === "number",
"legacy pending memory should receive a retention clock when promoted");
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
});
test("session.compacted reinforces existing exact workspace memory", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-test-"));
try {
const now = new Date().toISOString();
const oldRetentionClock = Date.now() - 10 * 24 * 60 * 60 * 1000;
await updateWorkspaceMemory(tmpDir, store => {
store.entries.push({
id: "existing-memory",
type: "decision",
text: "Use frozen rendered snapshots for cache stability.",
source: "explicit",
confidence: 1,
status: "active",
createdAt: now,
updatedAt: now,
retentionClock: oldRetentionClock,
});
return store;
});
await saveSessionState(tmpDir, {
version: 1,
sessionID: "reinforce-existing-session",
turn: 1,
updatedAt: now,
activeFiles: [],
openErrors: [],
recentDecisions: [],
pendingMemories: [{
id: "pending-duplicate",
type: "decision",
text: "Use frozen rendered snapshots for cache stability.",
source: "explicit",
confidence: 1,
status: "active",
createdAt: now,
updatedAt: now,
}],
});
const plugin = await MemoryV2Plugin({ directory: tmpDir, client: mockRootClient() });
await (plugin as Record<string, Function>)["event"]({
event: { type: "session.compacted", properties: { sessionID: "reinforce-existing-session" } },
});
const workspace = await loadWorkspaceMemory(tmpDir);
const existing = workspace.entries.find(entry => entry.id === "existing-memory");
assert.equal(workspace.entries.filter(entry => /frozen rendered/.test(entry.text)).length, 1);
assert.equal(existing?.reinforcementCount, 1);
assert.equal(existing?.lastReinforcedSessionID, "reinforce-existing-session");
assert.ok((existing?.retentionClock ?? 0) > oldRetentionClock);
assert.equal((await loadSessionState(tmpDir, "reinforce-existing-session")).pendingMemories.length, 0);
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
@@ -996,85 +813,6 @@ test("integration: explicit memory flows from user message through pending journ
}
});
test("explicit memory lifecycle emits detected appended and promoted evidence", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-test-"));
try {
const plugin = await MemoryV2Plugin({
directory: tmpDir,
client: mockClientWithLatestUser("remember this: Prefer evidence-backed lifecycle tests.", "msg-evidence-life"),
});
await (plugin as Record<string, Function>)["experimental.chat.system.transform"](
{ sessionID: "evidence-life-session", model: {} },
{ system: ["base header"] },
);
await (plugin as Record<string, Function>)["event"]({
event: { type: "session.compacted", properties: { sessionID: "evidence-life-session" } },
});
const events = await queryEvidenceEvents(tmpDir, { newestFirst: false });
const eventTypes = events.map(event => event.type);
assert.ok(eventTypes.includes("explicit_memory_detected"));
assert.ok(eventTypes.includes("pending_memory_appended"));
assert.ok(eventTypes.includes("promotion_promoted"));
const output = { system: ["base header"] };
await (plugin as Record<string, Function>)["experimental.chat.system.transform"](
{ sessionID: "evidence-life-session", model: {} },
output,
);
const finalEvents = await queryEvidenceEvents(tmpDir, { newestFirst: false });
assert.ok(finalEvents.map(event => event.type).includes("render_selected"));
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
});
test("session.compacted promotes first-time explicit memory without self-reinforcement", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-test-"));
try {
const plugin = await MemoryV2Plugin({
directory: tmpDir,
client: mockClientWithLatestUser(
"remember this: Prefer no self reinforcement during first promotion.",
"msg-no-self-reinforce",
),
});
await (plugin as Record<string, Function>)["experimental.chat.system.transform"](
{ sessionID: "no-self-reinforce-session", model: {} },
{ system: ["base header"] },
);
await (plugin as Record<string, Function>)["event"]({
event: {
type: "session.compacted",
properties: { sessionID: "no-self-reinforce-session" },
},
});
const workspace = await loadWorkspaceMemory(tmpDir);
const promoted = workspace.entries.find(entry =>
/Prefer no self reinforcement/.test(entry.text)
);
assert.ok(promoted, "explicit memory should be promoted");
assert.equal(promoted.reinforcementCount ?? 0, 0);
assert.equal(promoted.lastReinforcedSessionID, undefined);
const state = await loadSessionState(tmpDir, "no-self-reinforce-session");
assert.equal(state.pendingMemories.length, 0);
assert.equal((await loadPendingJournal(tmpDir)).entries.length, 0);
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
});
test("integration: compaction candidate flows through journal promotion and clears pending journal", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "memory-plugin-test-"));
@@ -1684,9 +1422,9 @@ test("session.compacted clears compaction pending memory rejected by workspace e
try {
const now = new Date().toISOString();
await updateWorkspaceMemory(tmpDir, store => {
for (let i = 0; i < 10; i += 1) {
for (let i = 0; i < 28; i += 1) {
store.entries.push({
id: `mem_high_feedback_${i}`,
id: `mem_high_${i}`,
type: "feedback",
text: `High priority user feedback memory ${i} that should outrank low priority references.`,
source: "explicit",
@@ -1696,30 +1434,6 @@ test("session.compacted clears compaction pending memory rejected by workspace e
updatedAt: now,
});
}
for (let i = 0; i < 10; i += 1) {
store.entries.push({
id: `mem_high_decision_${i}`,
type: "decision",
text: `High priority decision memory ${i} that should outrank low priority references.`,
source: "explicit",
confidence: 1,
status: "active",
createdAt: now,
updatedAt: now,
});
}
for (let i = 0; i < 8; i += 1) {
store.entries.push({
id: `mem_high_project_${i}`,
type: "project",
text: `High priority project memory ${i} that should outrank low priority references.`,
source: "explicit",
confidence: 1,
status: "active",
createdAt: now,
updatedAt: now,
});
}
return store;
});
@@ -1762,9 +1476,9 @@ test("session.compacted keeps explicit pending memory rejected by workspace entr
try {
const now = new Date().toISOString();
await updateWorkspaceMemory(tmpDir, store => {
for (let i = 0; i < 10; i += 1) {
for (let i = 0; i < 28; i += 1) {
store.entries.push({
id: `mem_high_explicit_reject_feedback_${i}`,
id: `mem_high_explicit_reject_${i}`,
type: "feedback",
text: `Pinned high priority feedback for explicit reject ${i}.`,
source: "explicit",
@@ -1774,30 +1488,6 @@ test("session.compacted keeps explicit pending memory rejected by workspace entr
updatedAt: now,
});
}
for (let i = 0; i < 10; i += 1) {
store.entries.push({
id: `mem_high_explicit_reject_decision_${i}`,
type: "decision",
text: `Pinned high priority decision for explicit reject ${i}.`,
source: "explicit",
confidence: 1,
status: "active",
createdAt: now,
updatedAt: now,
});
}
for (let i = 0; i < 8; i += 1) {
store.entries.push({
id: `mem_high_explicit_reject_project_${i}`,
type: "project",
text: `Pinned high priority project for explicit reject ${i}.`,
source: "explicit",
confidence: 1,
status: "active",
createdAt: now,
updatedAt: now,
});
}
return store;
});
@@ -1841,9 +1531,9 @@ test("explicit capacity rejection records bounded retry metadata", async () => {
try {
const now = new Date().toISOString();
await updateWorkspaceMemory(tmpDir, store => {
for (let i = 0; i < 10; i += 1) {
for (let i = 0; i < 28; i += 1) {
store.entries.push({
id: `mem_high_bounded_reject_feedback_${i}`,
id: `mem_high_bounded_reject_${i}`,
type: "feedback",
text: `Pinned high priority feedback for bounded rejection ${i}.`,
source: "explicit",
@@ -1853,30 +1543,6 @@ test("explicit capacity rejection records bounded retry metadata", async () => {
updatedAt: now,
});
}
for (let i = 0; i < 10; i += 1) {
store.entries.push({
id: `mem_high_bounded_reject_decision_${i}`,
type: "decision",
text: `Pinned high priority decision for bounded rejection ${i}.`,
source: "explicit",
confidence: 1,
status: "active",
createdAt: now,
updatedAt: now,
});
}
for (let i = 0; i < 8; i += 1) {
store.entries.push({
id: `mem_high_bounded_reject_project_${i}`,
type: "project",
text: `Pinned high priority project for bounded rejection ${i}.`,
source: "explicit",
confidence: 1,
status: "active",
createdAt: now,
updatedAt: now,
});
}
return store;
});
@@ -1948,9 +1614,9 @@ test("session.compacted clears compaction pending memories when all rejected by
try {
const now = new Date().toISOString();
await updateWorkspaceMemory(tmpDir, store => {
for (let i = 0; i < 10; i += 1) {
for (let i = 0; i < 28; i += 1) {
store.entries.push({
id: `mem_high_all_rejected_feedback_${i}`,
id: `mem_high_all_rejected_${i}`,
type: "feedback",
text: `Pinned high priority feedback ${i} that keeps the workspace entry cap full.`,
source: "explicit",
@@ -1960,30 +1626,6 @@ test("session.compacted clears compaction pending memories when all rejected by
updatedAt: now,
});
}
for (let i = 0; i < 10; i += 1) {
store.entries.push({
id: `mem_high_all_rejected_decision_${i}`,
type: "decision",
text: `Pinned high priority decision ${i} that keeps the workspace entry cap full.`,
source: "explicit",
confidence: 1,
status: "active",
createdAt: now,
updatedAt: now,
});
}
for (let i = 0; i < 8; i += 1) {
store.entries.push({
id: `mem_high_all_rejected_project_${i}`,
type: "project",
text: `Pinned high priority project ${i} that keeps the workspace entry cap full.`,
source: "explicit",
confidence: 1,
status: "active",
createdAt: now,
updatedAt: now,
});
}
return store;
});
+12 -49
View File
@@ -1,7 +1,7 @@
import test from "node:test";
import assert from "node:assert/strict";
import type { LongTermMemoryEntry } from "../src/types.ts";
import { accountPendingPromotions, promotionAccountingEvidenceEvents } from "../src/promotion-accounting.ts";
import { accountPendingPromotions } from "../src/promotion-accounting.ts";
import { memoryKey } from "../src/pending-journal.ts";
import type { MemoryConsolidationEvent } from "../src/workspace-memory.ts";
import { workspaceMemoryExactKey, workspaceMemoryIdentityKey } from "../src/workspace-memory.ts";
@@ -230,56 +230,19 @@ test("accountPendingPromotions marks manual capacity rejection as retryable", ()
assert.deepEqual([...result.retryableRejectedKeys], [memoryKey(pending[0])]);
});
test("promotionAccountingEvidenceEvents maps every promotion outcome with relations", () => {
const promoted = mem("promoted", "Promoted memory should produce evidence.", { source: "explicit" });
const absorbed = mem("absorbed", "Absorbed memory should produce evidence.", { source: "explicit" });
const retained = mem("retained", "absorbed memory should produce evidence.", { source: "explicit" });
const identityAbsorbed = mem("identity-absorbed", "Project config lives in `src/config.ts`", { type: "reference" });
const identityRetained = mem("identity-retained", "Project config lives in `./src/config.ts`", { type: "reference" });
const superseded = mem("superseded", "Parser supports 3 formats.", { source: "compaction" });
const replacement = mem("replacement", "Parser supports 4 formats.", { source: "compaction" });
const capacity = mem("capacity", "Capacity rejected explicit memory should retry.", { source: "explicit", type: "reference" });
const exhausted = mem("exhausted", "Exhausted explicit memory should stop retrying.", { source: "explicit", type: "reference" });
const pending = [promoted, absorbed, identityAbsorbed, superseded, capacity, exhausted];
const accounting = {
promotedKeys: new Set([memoryKey(promoted)]),
absorbedKeys: new Set([memoryKey(absorbed), memoryKey(identityAbsorbed)]),
supersededKeys: new Set([memoryKey(superseded)]),
rejectedKeys: new Set([memoryKey(capacity), memoryKey(exhausted)]),
retryableRejectedKeys: new Set([memoryKey(capacity), memoryKey(exhausted)]),
clearableKeys: new Set([memoryKey(promoted), memoryKey(absorbed), memoryKey(identityAbsorbed), memoryKey(superseded), memoryKey(exhausted)]),
};
const events = [
{ ...event(absorbed, "absorbed_exact"), retainedId: retained.id },
{ ...event(identityAbsorbed, "absorbed_identity"), retainedId: identityRetained.id },
{ ...event(superseded, "superseded_existing"), retainedId: replacement.id, supersededId: superseded.id },
event(capacity, "rejected_capacity"),
event(exhausted, "rejected_capacity"),
];
test("accountPendingPromotions clears compaction stale rejection from accounting", () => {
const pending = [mem("pending_stale", "Stale compaction reference should be terminal.", {
type: "reference",
source: "compaction",
})];
const evidence = promotionAccountingEvidenceEvents({
const result = accountPendingPromotions({
pending,
after: [promoted, retained, identityRetained, replacement],
events,
accounting,
exhaustedRejectedKeys: new Set([memoryKey(exhausted)]),
before: [],
after: [],
events: [event(pending[0], "rejected_stale")],
});
const expectedPromotionEventTypes = new Set([
"promotion_promoted",
"promotion_absorbed_exact",
"promotion_absorbed_identity",
"promotion_superseded",
"promotion_rejected_capacity",
"promotion_retry_scheduled",
"promotion_retry_exhausted",
]);
assert.deepEqual(new Set(evidence.map(event => event.type)), expectedPromotionEventTypes);
const absorbedEvent = evidence.find(event => event.type === "promotion_absorbed_exact");
assert.ok(absorbedEvent?.relations?.some(relation => relation.role === "absorbed" && relation.memory?.memoryId === absorbed.id));
assert.ok(absorbedEvent?.relations?.some(relation => relation.role === "retained" && relation.memory?.memoryId === retained.id));
const supersededEvent = evidence.find(event => event.type === "promotion_superseded");
assert.ok(supersededEvent?.relations?.some(relation => relation.role === "superseded" && relation.memory?.memoryId === superseded.id));
assert.ok(supersededEvent?.relations?.some(relation => relation.role === "superseded_by" && relation.memory?.memoryId === replacement.id));
assert.deepEqual([...result.rejectedKeys], [memoryKey(pending[0])]);
assert.deepEqual([...result.clearableKeys], [memoryKey(pending[0])]);
});
+3 -124
View File
@@ -1,13 +1,11 @@
import test from "node:test";
import assert from "node:assert/strict";
import { existsSync } from "node:fs";
import { mkdir, mkdtemp, readdir, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { spawn } from "node:child_process";
import { readJSON, updateJSON } from "../src/storage.ts";
import { queryEvidenceEvents } from "../src/evidence-log.ts";
import { workspaceMemoryPath } from "../src/paths.ts";
import { updateJSON } from "../src/storage.ts";
test("updateJSON serializes concurrent increments", async () => {
const root = await mkdtemp(join(tmpdir(), "wm-storage-"));
@@ -39,44 +37,6 @@ test("updateJSON does not replace corrupt JSON with fallback", async () => {
}
});
test("readJSON quarantines corrupt JSON and returns fallback", async () => {
const dir = await mkdtemp(join(tmpdir(), "wm-storage-corrupt-"));
const path = join(dir, "store.json");
try {
await writeFile(path, "{ invalid json", "utf8");
const loaded = await readJSON(path, () => ({ ok: true }));
assert.deepEqual(loaded, { ok: true });
const files = await readdir(dir);
assert.equal(files.includes("store.json"), false);
assert.equal(files.some(file => file.startsWith("store.json.corrupt-")), true);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("readJSON emits corrupt JSON quarantine evidence for workspace stores", async () => {
const root = await mkdtemp(join(tmpdir(), "wm-storage-evidence-corrupt-"));
try {
const path = await workspaceMemoryPath(root);
await mkdir(dirname(path), { recursive: true });
await writeFile(path, "{ invalid json", "utf8");
const loaded = await readJSON(path, () => ({ ok: true }));
const events = await queryEvidenceEvents(root, { types: ["storage_corrupt_json_quarantined"] });
assert.deepEqual(loaded, { ok: true });
assert.equal(events.length, 1);
assert.equal(events[0].reasonCodes.includes("invalid_json"), true);
assert.equal(JSON.stringify(events).includes("invalid json"), false);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("updateJSON recovers stale lock files left by crashed process", async () => {
const root = await mkdtemp(join(tmpdir(), "wm-storage-stale-lock-"));
try {
@@ -92,47 +52,6 @@ test("updateJSON recovers stale lock files left by crashed process", async () =>
}
});
test("updateJSON emits stale lock recovery evidence for workspace stores", async () => {
const root = await mkdtemp(join(tmpdir(), "wm-storage-evidence-stale-lock-"));
try {
const path = await workspaceMemoryPath(root);
const lockPath = `${path}.lock`;
await mkdir(dirname(path), { recursive: true });
await writeFile(lockPath, `999999\n0\n`, "utf8");
await updateJSON(path, () => ({ count: 0 }), current => ({ count: current.count + 1 }));
const events = await queryEvidenceEvents(root, { types: ["storage_stale_lock_recovered"] });
assert.equal(events.length, 1);
assert.equal(events[0].reasonCodes.includes("stale_lock"), true);
assert.equal(JSON.stringify(events).includes("999999"), false);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("updateJSON emits lock timeout evidence and still throws", async () => {
const root = await mkdtemp(join(tmpdir(), "wm-storage-evidence-timeout-"));
try {
const path = await workspaceMemoryPath(root);
const lockPath = `${path}.lock`;
await mkdir(dirname(path), { recursive: true });
await writeFile(lockPath, `${process.pid}\n${Date.now()}\n`, "utf8");
await assert.rejects(
updateJSON(path, () => ({ count: 0 }), current => current),
/Timed out waiting for lock/,
);
const events = await queryEvidenceEvents(root, { types: ["storage_lock_timeout"] });
assert.equal(events.length, 1);
assert.equal(events[0].reasonCodes.includes("lock_wait_timeout"), true);
assert.equal(JSON.stringify(events).includes(String(process.pid)), false);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("updateJSON serializes writes across separate node processes", async () => {
const root = await mkdtemp(join(tmpdir(), "wm-storage-xproc-"));
try {
@@ -162,43 +81,3 @@ test("updateJSON serializes writes across separate node processes", async () =>
await rm(root, { recursive: true, force: true });
}
});
test("updateJSON waits for a live cross-process lock and preserves both updates", async () => {
const root = await mkdtemp(join(tmpdir(), "wm-storage-live-lock-"));
try {
const path = join(root, "counter.json");
const worker = `
import { updateJSON } from ${JSON.stringify(new URL("../src/storage.ts", import.meta.url).href)};
const path = process.argv[1];
await updateJSON(path, () => ({ count: 0, order: [] }), async current => {
await new Promise(resolve => setTimeout(resolve, 250));
return { count: current.count + 1, order: [...current.order, "child"] };
});
`;
const child = spawn(
process.execPath,
["--experimental-strip-types", "--input-type=module", "-e", worker, path],
{ stdio: "inherit" },
);
await new Promise(resolve => setTimeout(resolve, 50));
await updateJSON(path, () => ({ count: 0, order: [] as string[] }), current => ({
count: current.count + 1,
order: [...current.order, "parent"],
}));
await new Promise<void>((resolve, reject) => {
child.on("exit", code => code === 0 ? resolve() : reject(new Error(`child exited ${code}`)));
child.on("error", reject);
});
const final = await updateJSON(path, () => ({ count: 0, order: [] as string[] }), current => current);
assert.equal(final.count, 2);
assert.deepEqual(new Set(final.order), new Set(["child", "parent"]));
assert.equal(existsSync(`${path}.lock`), false);
} finally {
await rm(root, { recursive: true, force: true });
}
});
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -25,6 +25,6 @@
"allowImportingTsExtensions": true,
"noEmit": true
},
"include": ["index.ts", "src/**/*.ts", "scripts/memory-diag.ts", "scripts/memory-diag/**/*.ts"],
"include": ["index.ts", "src/**/*.ts"],
"exclude": ["node_modules", "dist"]
}