diff --git a/AGENTS.md b/AGENTS.md index 144a373..ab1d3e5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,11 +2,11 @@ ## Project Overview -The **OpenCode Working Memory Plugin** provides a four-tier memory architecture for AI agents: -- **Core Memory** - Persistent blocks (goal/progress/context) that survive compaction -- **Working Memory** - Session-scoped context with slots (error/decision/todo/dependency) and memory pool -- **Smart Pruning** - Automatic filtering of tool outputs before adding to context -- **Pressure Monitoring** - Tracks context usage and triggers interventions at thresholds +The **OpenCode Working Memory Plugin** provides a **three-layer memory architecture** for AI agents: + +1. **Workspace Memory** - Long-term memory that persists across sessions (decisions, project info, references) +2. **Hot Session State** - Automatic tracking of active files, open errors, and recent decisions +3. **Native OpenCode State** - Delegated to OpenCode's built-in todos during compaction Written in **TypeScript** for the OpenCode agent environment. @@ -17,6 +17,8 @@ Written in **TypeScript** for the OpenCode agent environment. git clone https://github.com/sdwolf4103/opencode-working-memory.git cd opencode-working-memory npm install +npm test +npm run typecheck # For usage (see README.md) ``` @@ -30,24 +32,45 @@ npx tsc --noEmit ``` ### Testing -Tests are manually verified through OpenCode sessions: ```bash -# 1. Load plugin in OpenCode session -# 2. Run commands that trigger hooks (e.g., tool execution, compaction) -# 3. Inspect .opencode/memory-core/ and .opencode/memory-working/ -# 4. Verify memory blocks appear in system prompts +# Run all tests +npm test + +# Run specific test file +npx node --test --experimental-strip-types tests/plugin.test.ts ``` +Tests verify: +- Error extraction false positive guards +- Explicit memory trigger patterns +- Negative memory request filtering +- Compaction candidate quality gate +- Workspace memory rendering +- Session state tracking + ### File Structure ``` opencode-working-memory/ -├── index.ts # Main plugin (1700+ lines) -├── package.json # Plugin manifest -├── tsconfig.json # TypeScript config -├── LICENSE # MIT license -├── README.md # User documentation -├── AGENTS.md # This file (developer guide) -└── docs/ # Detailed documentation +├── index.ts # Plugin entry point (exports PluginModule) +├── src/ +│ ├── plugin.ts # Main plugin implementation +│ ├── extractors.ts # Memory extraction logic +│ ├── workspace-memory.ts # Workspace memory management +│ ├── session-state.ts # Session state tracking +│ ├── storage.ts # File storage utilities +│ ├── paths.ts # Path utilities +│ ├── opencode.ts # OpenCode SDK types +│ └── types.ts # Type definitions +├── tests/ +│ ├── plugin.test.ts # Plugin hook tests +│ ├── extractors.test.ts # Extractor tests +│ └── workspace-memory.test.ts # Workspace memory tests +├── package.json # Plugin manifest +├── tsconfig.json # TypeScript config +├── LICENSE # MIT license +├── README.md # User documentation +├── AGENTS.md # This file (developer guide) +└── docs/ # Detailed documentation ├── installation.md ├── architecture.md └── configuration.md @@ -59,39 +82,38 @@ opencode-working-memory/ ```typescript // ✅ REQUIRED: Full type annotations, no implicit any -async function loadCoreMemory( - directory: string, - sessionID: string -): Promise +async function loadWorkspaceMemory( + workspaceKey: string +): Promise // ❌ AVOID: Implicit any types -async function loadCoreMemory(directory, sessionID) { } +async function loadWorkspaceMemory(workspaceKey) { } ``` ### Type Definitions ```typescript // ✅ REQUIRED: Define types at module top -type CoreMemory = { - sessionID: string; - blocks: { - goal: CoreBlock; - progress: CoreBlock; - context: CoreBlock; - }; +export type LongTermMemoryEntry = { + id: string; + type: LongTermType; + text: string; + source: LongTermSource; + confidence: number; + status: "active" | "superseded"; + createdAt: string; updatedAt: string; }; // ✅ USE: Union types for variants (not enums) -type PressureLevel = "safe" | "moderate" | "high" | "critical"; +export type LongTermType = "feedback" | "project" | "decision" | "reference"; +export type LongTermSource = "explicit" | "compaction" | "manual"; -// ✅ USE: Record<> for keyed configs -const SLOT_CONFIG: Record = { - error: 3, - decision: 5, - todo: 3, - dependency: 3, -}; +// ✅ USE: const assertions for limits +export const LONG_TERM_LIMITS = { + maxRenderedChars: 5200, + maxEntries: 28, +} as const; ``` ### Imports & Module Organization @@ -105,57 +127,52 @@ import { join } from "path"; // 2. Third-party (OpenCode SDK) import type { Plugin } from "@opencode-ai/plugin"; -import { tool } from "@opencode-ai/plugin"; -// 3. Local modules (if any) -// (none currently) +// 3. Local modules +import { loadWorkspaceMemory } from "./storage.js"; ``` ### Naming Conventions ```typescript // ✅ REQUIRED: camelCase for variables & functions -const maxItems = 50; -async function loadCoreMemory() { } +const maxEntries = 28; +async function loadWorkspaceMemory() { } // ✅ REQUIRED: SCREAMING_SNAKE_CASE for constants -const CORE_MEMORY_LIMITS = { goal: 1000, progress: 2000, context: 1500 }; -const SLOT_CONFIG = { error: 3, decision: 5, todo: 3, dependency: 3 }; +const LONG_TERM_LIMITS = { maxRenderedChars: 5200, maxEntries: 28 }; +const HOT_STATE_LIMITS = { maxRenderedChars: 1200 }; // ✅ REQUIRED: PascalCase for types -type CoreMemory = { ... }; -type WorkingMemoryItem = { ... }; +type WorkspaceMemoryStore = { ... }; +type SessionState = { ... }; -// ✅ REQUIRED: get*/set*/load*/save* naming for file operations -function getCoreMemoryPath(directory: string, sessionID: string): string { } -async function loadCoreMemory(directory: string, sessionID: string): Promise { } -async function saveCoreMemory(directory: string, memory: CoreMemory): Promise { } - -// ✅ REQUIRED: ensure*/validate* for pre-checks -async function ensureCoreMemoryDir(directory: string): Promise { } - -// ✅ REQUIRED: Prefix private/internal functions with _ -function _compressPath(filePath: string): string { } +// ✅ REQUIRED: get*/load*/save* naming for file operations +function getWorkspaceMemoryPath(workspaceKey: string): string { } +async function loadWorkspaceMemory(workspaceKey: string): Promise { } +async function saveWorkspaceMemory(memory: WorkspaceMemoryStore): Promise { } ``` ### Function Signatures & Organization ```typescript // ✅ REQUIRED: Parameters on separate lines if > 80 chars -async function loadWorkingMemory( - directory: string, - sessionID: string -): Promise { +async function extractWorkspaceMemoryCandidates( + content: string +): Promise { // ... } // ✅ REQUIRED: Explicit return types (no inference) -function getCompactionLogPath(directory: string, sessionID: string): string { - return join(directory, ".opencode", "memory-working", `${sessionID}_compaction.json`); +function renderWorkspaceMemory( + entries: LongTermMemoryEntry[], + maxChars: number +): string { + // ... } // ✅ REQUIRED: Async for file/network I/O -async function saveCoreMemory(directory: string, memory: CoreMemory): Promise { +async function saveWorkspaceMemory(memory: WorkspaceMemoryStore): Promise { // ... } ``` @@ -163,28 +180,23 @@ async function saveCoreMemory(directory: string, memory: CoreMemory): Promise { - const path = getCoreMemoryPath(directory, sessionID); +// ✅ REQUIRED: Try-catch with graceful degradation +async function loadWorkspaceMemory(workspaceKey: string): Promise { + const path = getWorkspaceMemoryPath(workspaceKey); if (!existsSync(path)) return null; try { const content = await readFile(path, "utf-8"); - return JSON.parse(content) as CoreMemory; + return JSON.parse(content) as WorkspaceMemoryStore; } catch (error) { - console.error("Failed to load core memory:", error); - return null; // Graceful degradation + // Plugin should never block agent - return null and continue + return null; } } -// ✅ REQUIRED: Type guards for runtime safety -if (!existsSync(path)) { - return null; -} - // ✅ REQUIRED: Validate JSON before use const data = JSON.parse(content); -const typedData = data as CoreMemory; // Explicit cast after validation +const typedData = data as WorkspaceMemoryStore; // Explicit cast after validation ``` ### Comments & Documentation @@ -192,149 +204,123 @@ const typedData = data as CoreMemory; // Explicit cast after validation ```typescript // ✅ REQUIRED: Section headers for major sections // ============================================================================ -// Phase 1: Core Memory Foundation +// Workspace Memory: Long-term cross-session storage // ============================================================================ // ✅ REQUIRED: Block comments for complex logic -// Migration: Convert old format (items array) to new format (slots + pool) -if (data.items && !data.slots) { - // ... migration logic +// Quality gate: Reject candidates that are git hashes, errors, or path-heavy +function shouldAcceptWorkspaceMemoryCandidate(candidate: string): boolean { + // ... } -// ✅ USE: Inline comments sparingly -const gamma = 0.85; // Exponential decay rate (15% per event) - -// ✅ AVOID: Over-commenting obvious code -const name = "test"; // Set name to test ❌ (obvious) -``` - -### Code Organization - -```typescript -// ✅ REQUIRED: Organize plugin file by phase/feature -// 1. Header & module documentation -// 2. Imports -// 3. Types & schemas (grouped by phase) -// 4. Constants & configs -// 5. Helper functions (private first, public after) -// 6. Main plugin export -// 7. Hook implementations - -export default { - // Plugin definition -} as Plugin; -``` - -### Working with OpenCode Plugin SDK - -```typescript -// ✅ REQUIRED: Use proper hook signatures -import { tool, type Plugin } from "@opencode-ai/plugin"; - -export default { - id: "working-memory", - name: "Working Memory Plugin", - - // ✅ Core hooks - hooks: { - "tool.execute.after": async (ctx) => { - // Tool just executed - }, - "experimental.chat.system.transform": async (ctx) => { - // Transform system prompt before sending - }, - "experimental.session.compacting": async (ctx) => { - // Session is being compacted (clearing old messages) - }, - }, - - // ✅ Exposed tools - tools: [ - tool({ - id: "core_memory_update", - name: "Update Core Memory", - description: "Update goal/progress/context blocks", - // ... schema & execute - }), - ], -} as Plugin; +// ✅ USE: Inline comments sparingly for non-obvious logic +const canonical = normalizeText(text); // Lowercase, strip punctuation, collapse whitespace ``` ## Key Implementation Details -### Core Memory Files -- Location: `.opencode/memory-core/.json` -- Schema: `{ sessionID, blocks: { goal, progress, context }, updatedAt }` -- Limits: goal (1000 chars), progress (2000 chars), context (1500 chars) +### Plugin Entry Point -### Working Memory Files -- Location: `.opencode/memory-working/.json` -- Schema: `{ sessionID, slots, pool, eventCounter, updatedAt }` -- Slot limits: error (3), decision (5), todo (3), dependency (3) -- Pool decay: γ=0.85 per event +```typescript +// index.ts +import { MemoryV2Plugin } from "./src/plugin.ts"; -### Pressure Monitoring -- Triggers at: 70% (safe→moderate), 85% (moderate→high), 95% (high→critical) -- Files: `.opencode/memory-working/_pressure.json` -- Intervention: Sends `promptAsync()` with complete visible prompt +export default { + id: "working-memory", + server: MemoryV2Plugin, +}; +``` -### Storage Governance (Layer 1 & 2) -- **Layer 1**: Session deletion cleanup - removes orphaned memory files -- **Layer 2**: Tool output cache sweep - maintains 300 most recent files, 7-day TTL -- Triggered at `eventCounter % 500 === 0` (automatic maintenance) +### Workspace Memory Files + +- **Location**: `~/.local/share/opencode-working-memory/workspaces/{workspaceKey}/workspace-memory.json` +- **Workspace Key**: First 16 chars of `sha256(realpath(workspaceRoot))` +- **Schema**: See `src/types.ts:WorkspaceMemoryStore` +- **Limits**: 5200 chars, 28 entries max + +### Session State Files + +- **Location**: `~/.local/share/opencode-working-memory/workspaces/{workspaceKey}/sessions/{sessionID}.json` +- **Session ID**: Hash of session ID from OpenCode +- **Schema**: See `src/types.ts:SessionState` + +### Memory Types + +| Type | Purpose | Stale After | +|------|---------|-------------| +| `feedback` | User preferences | 90 days | +| `project` | Project info | 60 days | +| `decision` | Important decisions | 45 days | +| `reference` | Key references | 90 days | + +### Quality Guards + +1. **False Positive Error Prevention**: Commands with "error" in output but `exitCode === undefined` are not tracked as errors +2. **Negative Memory Filtering**: "don't remember" patterns are correctly interpreted +3. **Compaction Quality Gate**: Rejects git hashes, stack traces, path-heavy facts + +## Plugin Hooks + +### `prompt:before` + +Injects workspace memory and hot session state into system prompt. + +### `tool.execute.before` + +Tracks active files (read, grep, edit, write actions). + +### `tool.execute.after` + +- Tracks open errors from failed commands +- Clears errors when commands succeed +- Ignores `exitCode === undefined` + +### `compaction:before` + +Extracts workspace memory candidates from conversation, applies quality gate and deduplication. ## Debugging & Testing ### Manual Testing Steps -1. **Phase 1 (Core Memory)**: Check `.opencode/memory-core/` after `core_memory_update` -2. **Phase 2 (Smart Pruning)**: Verify tool outputs are filtered before context injection -3. **Phase 3 (Working Memory)**: Check `.opencode/memory-working/` for slot/pool items -4. **Phase 4 (Pressure Monitoring)**: Monitor pressure % in system prompts, verify interventions -5. **Phase 4.5 (Storage Governance)**: Run 500+ events, check sweep logs + +1. **Workspace Memory**: Check `~/.local/share/opencode-working-memory/workspaces/*/workspace-memory.json` after compaction +2. **Session State**: Check `~/.local/share/opencode-working-memory/workspaces/*/sessions/*.json` after tool usage +3. **Error Tracking**: Run failing commands, verify errors appear in session state +4. **Error Clearing**: Run successful commands, verify errors are cleared ### Common Issues -- **File not found**: Ensure `.opencode/` directory exists and is writable + +- **File not found**: Ensure `~/.local/share/opencode-working-memory/` exists and is writable - **Type errors**: Check all imports use `import type { ... }` for types -- **Lost memory**: Verify `.opencode/memory-*/` is in `.gitignore` (not committed) -- **Sweep not running**: Check `eventCounter` in `.json`, should trigger at multiples of 500 +- **Memory not persisting**: Verify workspace key is consistent (same workspace = same key) +- **False positive errors**: Check `exitCode` handling in `plugin.ts` ## Performance Considerations -- **Memory budgets**: Core (5.5k chars total), Working (1.6k chars for system prompt) -- **Pruning**: Hyper-aggressive mode activates at ≥85% pressure -- **Compaction**: Preserves most recent 10 items when space-constrained -- **Decay**: Pool items scored by exponential decay (γ=0.85) + mention count -- **Storage sweep**: Limits cache to 300 files, removes files older than 7 days - -## File Path References - -When referencing code locations in documentation/comments, use: -``` -path/to/file.ts:L123 or path/to/file.ts:Line 123 -``` - -Example: `Function sendPressureInterventionMessage() @ index.ts:L1286` +- **Workspace memory budget**: 5200 chars injected into system prompt +- **Session state budget**: 1200 chars injected into system prompt +- **Total overhead**: ~1500-6000 chars per message (minimal) +- **Storage footprint**: ~2-5 KB per workspace for memory, ~1-3 KB per session ## Contributing 1. Fork the repository 2. Create a feature branch: `git checkout -b feature/my-feature` 3. Make changes following the code style guidelines above -4. Test manually in OpenCode session -5. Commit with descriptive message: `git commit -m "Add feature: ..."` +4. Run tests: `npm test && npm run typecheck` +5. Commit with descriptive message: `git commit -m "feat: add ..."` 6. Push to your fork: `git push origin feature/my-feature` 7. Open a pull request ## Architecture Documentation See `docs/architecture.md` for detailed technical documentation including: -- Memory tier hierarchy -- Pruning algorithms -- Decay formulas -- Pressure monitoring logic -- Storage governance policies +- Three-layer memory architecture +- Memory extraction and quality gates +- Error fingerprinting +- Deduplication strategies --- -**Last Updated**: February 2026 -**Plugin Status**: Production (Phases 1-4.5 complete) +**Last Updated**: April 2026 +**Plugin Status**: Production (Memory V2 architecture) \ No newline at end of file diff --git a/README.md b/README.md index 44b2bef..d8a48e5 100644 --- a/README.md +++ b/README.md @@ -3,32 +3,19 @@ [![npm version](https://img.shields.io/npm/v/opencode-working-memory.svg)](https://www.npmjs.com/package/opencode-working-memory) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) -**Advanced four-tier memory architecture that keeps your AI agent sharp, focused, and never forgets what matters.** +**Automatic memory system that keeps your AI agent context-aware across compactions.** -Stop losing context across compactions. Stop watching your agent repeat the same mistakes. This plugin gives your OpenCode agent a professional-grade memory system that scales with your project complexity. +Stop losing context when OpenCode compacts your conversation. This plugin automatically tracks what matters — decisions, active files, open errors — and preserves it across sessions. ## What You Get -- 🧠 **Core Memory** - Persistent goal/progress/context blocks that survive compaction -- 💡 **Working Memory** - Smart slot-based system (errors, decisions, todos, dependencies) -- 🎯 **Memory Pressure Monitoring** - Real-time token tracking with automatic interventions -- 🧹 **Auto Storage Governance** - Prevents unbounded disk growth (300 files/session, 7-day TTL) -- 📊 **Smart Pruning** - Pressure-aware tool output compression -- ⚡ **Zero Configuration** - Works out of the box, highly customizable +- 🧠 **Workspace Memory** - Long-term memory that persists across sessions (decisions, project info, references) +- 📡 **Hot Session State** - Automatic tracking of active files, open errors, and recent decisions +- 🔧 **Smart Compaction** - Extracts and preserves important memories during compaction +- ⚡ **Zero Configuration** - Works out of the box, no tools to call ## Installation -**Option A: Let an AI agent do it** - -Paste this into any AI agent: - -``` -Install opencode-working-memory plugin by following: -https://raw.githubusercontent.com/sdwolf4103/opencode-working-memory/main/docs/installation.md -``` - -**Option B: Manual setup** - Add to your `~/.config/opencode/opencode.json`: ```json @@ -37,167 +24,156 @@ Add to your `~/.config/opencode/opencode.json`: } ``` -Restart OpenCode. The plugin is downloaded and installed automatically. - -### For LLM Agents - -1. Edit `~/.config/opencode/opencode.json` -2. Add `"opencode-working-memory"` to the `plugin` array -3. Verify the config file was saved correctly - -## Quick Start - -The plugin works automatically once installed. No configuration needed. - -**Try telling your agent:** - -``` -Use core_memory_update to set my current goal -``` -``` -Use core_memory_read to show me what you remember -``` -``` -Use working_memory_add to remember this file path -``` - -## Features - -### 🧠 Core Memory - -Persistent blocks that survive conversation resets: - -- **goal** (1000 chars) - Current task/objective -- **progress** (2000 chars) - What's done, in-progress, next steps -- **context** (1500 chars) - Key file paths, conventions, patterns - -### 💡 Working Memory - -Auto-extracts and ranks important information: - -- **Slots** (guaranteed visibility): errors, decisions, todos, dependencies -- **Pool** (ranked by relevance): file paths, recent activity -- Exponential decay keeps memory fresh -- FIFO limits prevent bloat - -### 🎯 Memory Pressure Monitoring - -Real-time token tracking from session database: - -- Monitors context window usage (75% moderate → 90% high) -- Proactive intervention messages when pressure is high -- Pressure-aware smart pruning (adapts compression based on pressure) - -### 🧹 Storage Governance - -Prevents unbounded disk growth: - -- Auto-cleanup on session deletion (all artifacts removed) -- Active cache management (max 300 files/session, 7-day TTL) -- Silent background operation - -### 📊 Smart Pruning - -Intelligent tool output compression: - -- Per-tool strategies (keep-all, keep-ends, keep-last, discard) -- Pressure-aware limits (2k/5k/10k lines based on memory pressure) -- Preserves important context while reducing noise - -## Documentation - -- [Installation Guide](docs/installation.md) - Detailed setup instructions -- [Architecture Overview](docs/architecture.md) - How it works under the hood -- [Configuration](docs/configuration.md) - Customization options -- [Agent Developer Guide](AGENTS.md) - For plugin developers - -## Tools Provided - -The plugin exposes these tools to your OpenCode agent: - -- `core_memory_update` - Update goal/progress/context blocks -- `core_memory_read` - Read current memory state -- `working_memory_add` - Manually add important items -- `working_memory_clear` - Clear all working memory -- `working_memory_clear_slot` - Clear specific slot (errors/decisions) -- `working_memory_remove` - Remove specific item by content +Restart OpenCode. The plugin activates automatically — no manual setup needed. ## How It Works +The plugin operates via three automatic layers: + ``` -┌───────────────────────────────────────────────────────────┐ -│ Core Memory (Always Visible) │ -│ ┌─────────┬──────────┬──────────┐ │ -│ │ Goal │ Progress │ Context │ │ -│ └─────────┴──────────┴──────────┘ │ -└───────────────────────────────────────────────────────────┘ - ↓ -┌───────────────────────────────────────────────────────────┐ -│ Working Memory (Auto-Extracted) │ -│ ┌──────────────────┬──────────────────┐ │ -│ │ Slots (FIFO) │ Pool (Ranked) │ │ -│ │ • errors │ • file-paths │ │ -│ │ • decisions │ • recent │ │ -│ │ • todos │ • mentions │ │ -│ │ • dependencies │ • decay score │ │ -│ └──────────────────┴──────────────────┘ │ -└───────────────────────────────────────────────────────────┘ - ↓ -┌───────────────────────────────────────────────────────────┐ -│ Memory Pressure Monitor │ -│ • Tracks tokens from session DB │ -│ • Warns at 75% (moderate) / 90% (high) │ -│ • Sends proactive interventions │ -│ • Adjusts pruning aggressiveness │ -└───────────────────────────────────────────────────────────┘ - ↓ -┌───────────────────────────────────────────────────────────┐ -│ Storage Governance │ -│ • Session deletion → cleanup all artifacts │ -│ • Every 20 calls → sweep old cache (300 max, 7d TTL) │ -│ • Silent background operation │ -└───────────────────────────────────────────────────────────┘ +┌─────────────────────────────────────────────────────────────┐ +│ WORKSPACE MEMORY (Long-term, cross-session) │ +│ Storage: ~/.local/share/opencode-working-memory/ │ +│ workspaces/{hash}/workspace-memory.json │ +│ │ +│ Types: feedback | project | decision | reference │ +│ Sources: explicit (user) | compaction | manual │ +│ Limits: 5200 chars / 28 entries │ +└─────────────────────────────────────────────────────────────┘ + ↓ +┌─────────────────────────────────────────────────────────────┐ +│ HOT SESSION STATE (Short-term, per-session) │ +│ Storage: ~/.local/share/opencode-working-memory/ │ +│ workspaces/{hash}/sessions/{sessionID}.json │ +│ │ +│ Tracks: │ +│ • Active files (read, grep, edit, write actions) │ +│ • Open errors (typecheck, test, lint, build, runtime) │ +│ • Recent decisions (auto-extracted) │ +└─────────────────────────────────────────────────────────────┘ + ↓ +┌─────────────────────────────────────────────────────────────┐ +│ NATIVE OPENCODE STATE │ +│ • Uses OpenCode's built-in todos during compaction │ +│ • No additional storage required │ +└─────────────────────────────────────────────────────────────┘ ``` -## Why This Plugin? +### Workspace Memory (Long-term) -**Without this plugin:** -- 🔴 Agent forgets context after compaction -- 🔴 Repeats resolved errors -- 🔴 Loses track of project structure -- 🔴 Context window fills up uncontrollably -- 🔴 Disk space grows unbounded +Persists across sessions within the same workspace. Automatically extracted during compaction when the agent marks something with "remember" or "note": -**With this plugin:** -- ✅ Persistent memory across compactions -- ✅ Smart auto-extraction of important info -- ✅ Real-time pressure monitoring with interventions -- ✅ Automatic storage cleanup -- ✅ Pressure-aware compression -- ✅ Zero configuration, works immediately +``` + +- [decision] Use npm cache for plugin loading, not npm link +- [project] This repo uses opencode-agenthub plugin system +- [reference] Storage: ~/.local/share/opencode-working-memory/... + +``` -## Does working-memory system increase token usage? It depends. +**Memory types:** +- `feedback` - User preferences for this workspace +- `project` - Project-level information +- `decision` - Important decisions made +- `reference` - Key references (paths, patterns) -It depends on your workflow. +**Sources:** +- `explicit` - User explicitly said "remember this" (confidence: 1.0) +- `compaction` - Extracted during compaction (confidence: 0.75) +- `manual` - Added programmatically (confidence: varies) -- 🧹 **Clean Slate user** (for example, using DCP and frequently restarting sessions) - - ⚠️ Yes, it might add slight overhead. - - Because you keep starting fresh, automated memory persistence does not get enough time to pay off. +### Hot Session State (Short-term) -- 🚀 **Long Haul user** (staying in one session until token limits/compaction hit) - - ✅ This plugin is a token saver. - - Without it, compaction can cause the agent to lose the goal, forget active files, or make wrong assumptions, which creates correction loops. - - By preserving high-value context (Goals, Progress, Active Files), the agent inherits its previous state quickly. The small memory-prompt cost avoids the larger cost of the agent getting lost. +Automatically tracks current session context: -## Configuration (Optional) +- **Active Files**: What files you're working on (ranked by recency and action type) +- **Open Errors**: Errors that haven't been fixed yet (typecheck, test failures, etc.) +- **Recent Decisions**: Decisions made this session (candidates for long-term promotion) -The plugin works great with zero configuration. To customize behavior, modify the constants at the top of `index.ts`. See the [Configuration Guide](docs/configuration.md) for all tunable options. +Injected into system prompt: +``` + +- [decision] Use npm cache for plugin loading... + + +--- + +- [project] This repo uses opencode-agenthub plugin system +- [decision] Memory V2 uses three-layer architecture + + +Active Files: +- src/plugin.ts (edit, 18x) +- tests/plugin.test.ts (edit, 5x) +- src/extractors.ts (grep, 3x) + +Open Errors: (none) +``` + +## Quality Guarantees + +The plugin includes several quality guards: + +- **No false positive errors**: Bash commands like `git log` or `cat` with "error" in output are not misidentified +- **Negative memory filtering**: "Don't remember this" is correctly interpreted +- **Compaction quality gate**: Rejects git hashes, stack traces, path-heavy facts from becoming long-term memories +- **Canonical deduplication**: Memories are deduplicated with case/punctuation normalization + +## No Tools Required + +Unlike other memory plugins, **this plugin has no manual tools**. Everything is automatic: + +- No `core_memory_update` — memory is extracted automatically +- No `core_memory_read` — memory is injected into system prompt +- No `working_memory_add` — active files are tracked automatically + +Just install and let it run. The plugin hooks into OpenCode's lifecycle events and does the right thing. + +## Configuration + +The plugin works out of the box with sensible defaults: + +- **Workspace Memory**: 5200 chars, 28 entries max +- **Hot State**: 1200 chars rendered, 8 active files, 3 errors shown +- **Storage**: `~/.local/share/opencode-working-memory/workspaces/{hash}/` + +See [Configuration Guide](docs/configuration.md) for customization options. + +## For AI Agents + +When using this plugin, the memory context appears in your system prompt. You can: + +1. **Tell users about memories**: "I remember you decided to use npm cache for plugins" +2. **Ask about preferences**: "Should I add this to my memory for this workspace?" +3. **Note important decisions**: These will be extracted during compaction + +To add something to long-term memory explicitly: +``` +Remember this: [your note here] +``` + +The plugin captures this during compaction. + +## Documentation + +- [Architecture Overview](docs/architecture.md) - How the three layers work +- [Configuration](docs/configuration.md) - Customization options +- [Installation Guide](docs/installation.md) - Step-by-step setup + +## Development + +```bash +git clone https://github.com/sdwolf4103/opencode-working-memory.git +cd opencode-working-memory +npm install +npm test +npm run typecheck +``` ## Requirements - OpenCode >= 1.0.0 - Node.js >= 18.0.0 -- `@opencode-ai/plugin` >= 1.2.0 ## License @@ -208,12 +184,6 @@ MIT License - see [LICENSE](LICENSE) file for details. - 📖 [Documentation](docs/) - 🐛 [Report Issues](https://github.com/sdwolf4103/opencode-working-memory/issues) -## Credits - -Inspired by the needs of real-world OpenCode usage and built to solve actual pain points in AI-assisted development. - -> This project is not affiliated with or endorsed by the OpenCode team. - --- -**Made with ❤️ for the OpenCode community** +**Made with ❤️ for the OpenCode community** \ No newline at end of file diff --git a/docs/architecture.md b/docs/architecture.md index 7aceb57..46dd71c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -2,374 +2,348 @@ ## Overview -The Working Memory Plugin implements a **four-tier memory architecture** designed to maximize context efficiency for AI agents in OpenCode sessions. +The Working Memory Plugin implements a **three-layer memory architecture** designed to preserve context across OpenCode session compactions. ``` ┌─────────────────────────────────────────────────────────────┐ -│ TIER 1: CORE MEMORY │ -│ Persistent blocks: goal (1000) | progress (2000) | context (1500) │ -│ Survives compaction, always visible in system prompt │ +│ LAYER 1: WORKSPACE MEMORY (Long-term, cross-session) │ +│ • Persistent storage: ~/.local/share/opencode-working-... │ +│ • Types: feedback | project | decision | reference │ +│ • Sources: explicit | compaction | manual │ +│ • Limits: 5200 chars / 28 entries │ +│ • Survives: session reset, workspace switch │ └─────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────┐ -│ TIER 2: WORKING MEMORY │ -│ Session-scoped slots + memory pool │ -│ Slots: error(3) | decision(5) | todo(3) | dependency(3) │ -│ Pool: Exponential decay (γ=0.85) + mention tracking │ +│ LAYER 2: HOT SESSION STATE (Short-term, per-session) │ +│ • Session-scoped tracking: active files, open errors │ +│ • Storage: sessions/{sessionID}.json │ +│ • Auto-extracted from tool usage patterns │ +│ • Cleared: on new session start │ └─────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────┐ -│ TIER 3: SMART PRUNING │ -│ Filters tool outputs before adding to conversation │ -│ Removes: file lists, verbose logs, repetitive content │ -│ Modes: normal → aggressive → hyper-aggressive │ -└─────────────────────────────────────────────────────────────┘ - ↓ -┌─────────────────────────────────────────────────────────────┐ -│ TIER 4: PRESSURE MONITORING │ -│ Tracks context usage: safe → moderate → high │ -│ Thresholds: 75% (moderate) | 90% (high) │ -│ Intervention: Sends promptAsync() with full visible prompt │ +│ LAYER 3: NATIVE OPENCODE STATE │ +│ • Uses OpenCode's built-in todos during compaction │ +│ • No additional storage required │ +│ • Delegated to OpenCode's native features │ └─────────────────────────────────────────────────────────────┘ ``` -## Phase 1: Core Memory Foundation +## Layer 1: Workspace Memory ### Purpose -Provide persistent memory blocks that survive conversation compaction and are always injected into the system prompt. + +Long-term memory that persists across sessions within the same workspace. Perfect for: +- Project conventions and patterns +- Important decisions that span sessions +- User preferences for this codebase ### Storage -- **Location**: `.opencode/memory-core/.json` + +- **Location**: `~/.local/share/opencode-working-memory/workspaces/{workspaceKey}/workspace-memory.json` +- **Workspace Key**: First 16 chars of `sha256(realpath(workspaceRoot))` - **Schema**: ```typescript { - sessionID: string; - blocks: { - goal: { content: string; chars: number; maxChars: 1000; updatedAt: string }; - progress: { content: string; chars: number; maxChars: 2000; updatedAt: string }; - context: { content: string; chars: number; maxChars: 1500; updatedAt: string }; - }; - updatedAt: string; + version: 1, + workspace: { root: string, key: string }, + limits: { maxRenderedChars: 5200, maxEntries: 28 }, + entries: LongTermMemoryEntry[], + updatedAt: string } ``` -### Character Limits -- **goal**: 1000 chars (ONE specific task) -- **progress**: 2000 chars (done/in-progress/blocked checklist) -- **context**: 1500 chars (current working files + key patterns) +### Entry Types -### Operations -- **replace**: Completely replace block content -- **append**: Add content to end (auto-adds newline) +| Type | Purpose | Example | +|------|---------|---------| +| `feedback` | User preferences | "User prefers functional React components" | +| `project` | Project-level info | "This monorepo uses turborepo" | +| `decision` | Important decisions | "Use PostgreSQL for primary database" | +| `reference` | Key references | "API endpoints defined in `src/api/`" | -### Tools -- `core_memory_update`: Update or append to blocks -- `core_memory_read`: Read current state of all blocks +### Entry Sources + +| Source | Confidence | How Added | +|--------|------------|-----------| +| `explicit` | 1.0 | User said "remember this" | +| `compaction` | 0.75 | Extracted during compaction | +| `manual` | varies | Programmatically added | + +### Memory Extraction + +During compaction, the plugin scans for `` blocks: + +``` + +- [decision] Use npm cache for plugin loading +- [project] This repo uses TypeScript with strict mode + +``` + +**Quality Gate**: Not all candidates become memories. The plugin rejects: +- Git commit hashes (e.g., `abc1234`) +- Raw errors (e.g., `Error: something failed`) +- Stack traces +- Path-heavy facts (>50% paths) +- Very short text (<20 chars) + +### Deduplication + +Memories are deduplicated using **canonical text matching**: +1. Normalize: lowercase, strip punctuation, collapse whitespace +2. Hash the canonical text +3. Keep the entry with highest confidence ### System Prompt Injection -Blocks are injected into every agent message as: + +Workspace memory is injected at the top of every message: + ``` - -... -... -... - + +- [decision] Use npm cache for plugin loading, not npm link +- [project] This repo uses opencode-agenthub plugin system +- [reference] Storage: ~/.local/share/opencode-working-memory/... + ``` -## Phase 2: Smart Pruning +## Layer 2: Hot Session State ### Purpose -Reduce context bloat by filtering tool outputs before they enter the conversation history. -### Pruning Modes - -#### Normal Mode (Pressure < 75%) -- Remove file/directory listings > 50 lines -- Truncate verbose tool outputs -- Keep first/last 30 lines of long outputs -- Preserve error messages and key information - -#### Aggressive Mode (75% ≤ Pressure < 90%) -- Threshold drops to 30 lines -- More aggressive truncation (first/last 20 lines) -- Filter repetitive content - -#### Hyper-Aggressive Mode (Pressure ≥ 90%) -- Threshold drops to 15 lines -- Keep only first/last 10 lines -- Maximum compression - -### Pruning Heuristics - -1. **File Listings**: Detect `ls`, `find`, `glob` outputs -2. **Directory Trees**: Detect tree-like structures with `/` -3. **Log Files**: Detect timestamp patterns, stack traces -4. **Repetitive Content**: Detect similar consecutive lines -5. **Synthetic Content**: Preserve `synthetic: true` markers - -### Implementation -Pruning happens in `tool.execute.after` hook before tool output enters conversation. - -## Phase 3: Working Memory - -### Purpose -Provide session-scoped memory with structured slots and a general-purpose pool with intelligent decay. +Track current session context automatically: +- What files are you working on? +- What errors are currently open? +- What decisions were made recently? ### Storage -- **Location**: `.opencode/memory-working/.json` + +- **Location**: `~/.local/share/opencode-working-memory/workspaces/{workspaceKey}/sessions/{hashedSessionID}.json` - **Schema**: ```typescript { - sessionID: string; - slots: { - error: Array; // Max 3 - decision: Array; // Max 5 - todo: Array; // Max 3 - dependency: Array; // Max 3 - }; - pool: Array; - eventCounter: number; - updatedAt: string; + version: 1, + sessionID: string, + turn: number, + updatedAt: string, + activeFiles: ActiveFile[], + openErrors: OpenError[], + recentDecisions: SessionDecision[] } ``` -### Slot Types +### Active Files -| Slot | Max Items | Purpose | -|------|-----------|---------| -| **error** | 3 | Recent errors that need fixing | -| **decision** | 5 | Important decisions made | -| **todo** | 3 | Current task checklist | -| **dependency** | 3 | File/package dependencies | +Automatically tracked from `tool.execute.before` events: -### Memory Pool +| Action | Ranking Boost | +|--------|---------------| +| `write` | 4x | +| `edit` | 3x | +| `read` | 2x | +| `grep` | 1x | -General-purpose storage with **exponential decay**: +Files are ranked by: `count * action_weight * recency_decay` -```typescript -score = exp(-γ * age) + mentionCount -``` +### Open Errors -Where: -- `γ = 0.85` (decay rate, 15% per event) -- `age = eventCounter - item.eventNumber` -- `mentionCount`: Number of times item mentioned in conversation +Tracked from `tool.execute.after` events when `exitCode !== 0`: -Items with `score < 0.01` are pruned. +| Category | Trigger Pattern | +|----------|-----------------| +| `typecheck` | `TS####:` or TypeScript errors | +| `test` | Test failures | +| `lint` | ESLint warnings/errors | +| `build` | Build failures | +| `runtime` | `Error:`, `TypeError:`, etc. | -### Auto-Extraction +**False Positive Guards**: +- Commands like `git log`, `cat` with "error" in output are ignored +- Only actual command failures (`exitCode !== 0`) trigger errors +- `exitCode === undefined` is treated as success (no error tracking) -Working memory items are **automatically extracted** from: -- Tool outputs (file paths, errors, dependencies) -- User messages (decisions, todos) -- Assistant responses (key information) +### Error Fingerprinting -### Manual Management +Errors are fingerprinted by: +1. Extract error message summary +2. Generate fingerprint: `first 12 chars of sha256(summary)` +3. Group similar errors by fingerprint -Tools: -- `working_memory_add`: Manually add item -- `working_memory_clear`: Clear all items -- `working_memory_clear_slot`: Clear specific slot (e.g., after fixing all errors) -- `working_memory_remove`: Remove specific item by content match +### recentDecisions + +Short-term decisions made this session. Candidates for promotion to workspace memory during compaction. ### System Prompt Injection +Hot session state is injected after workspace memory: + ``` - -Recent session context (auto-managed, sorted by relevance): +--- + +- [project] This repo uses TypeScript with strict mode + -⚠️ Errors: - - TypeError at line 42 in utils.ts - - Missing import in index.ts +Active Files: +- src/plugin.ts (edit, 18x) +- tests/plugin.test.ts (edit, 5x) -📁 Key Files: - - src/components/Button.tsx - - src/utils/helpers.ts - -(15 items shown, updated: 9:46:47 AM) - +Open Errors: (none) ``` -## Phase 4: Pressure Monitoring +## Layer 3: Native OpenCode State ### Purpose -Track conversation context usage and trigger interventions when approaching limits. -### Pressure Calculation +Delegate task tracking to OpenCode's native features. + +### Behavior + +- Uses OpenCode's built-in `todos` during compaction +- No additional storage or injection required +- Allows the agent to manage task lists natively + +## Plugin Hooks + +The plugin hooks into OpenCode lifecycle events: + +### `prompt:before` + +Injects workspace memory and hot session state into system prompt. + +### `tool.execute.before` + +Tracks active files (read, grep, edit, write actions). + +### `tool.execute.after` + +- Tracks open errors from failed commands +- Clears errors when commands succeed +- Ignores `exitCode === undefined` (successful commands without explicit exit codes) + +### `compaction:before` + +Extracts workspace memory candidates from conversation. +Applies quality gate, deduplication, and source priority. + +## Quality Guarantees + +### No False Positive Errors ```typescript -pressure = (visiblePromptChars / estimatedContextLimit) * 100 +// Bad: Would create false positive +"Error: something failed" in output + +// Good: Actually failed +exitCode === 1 && output.includes("Error") + +// Good: Actually succeeded +exitCode === 0 (clears errors for that category) + +// Good: Ignore ambiguous cases +exitCode === undefined → skip error tracking ``` -Where: -- `visiblePromptChars`: Total characters in system prompt + tool outputs -- `estimatedContextLimit`: ~180,000 chars (conservative estimate) - -### Pressure Levels - -| Level | Threshold | Behavior | -|-------|-----------|----------| -| **safe** | < 75% | Normal operation | -| **moderate** | 75-89% | Warning in system prompt + aggressive pruning | -| **high** | ≥ 90% | Hyper-aggressive pruning + intervention | - -### Pressure Storage - -- **Location**: `.opencode/memory-working/_pressure.json` -- **Schema**: - ```typescript - { - sessionID: string; - level: "safe" | "moderate" | "high"; - percentage: number; - visiblePromptChars: number; - estimatedLimit: 180000; - lastChecked: string; - interventionsSent: number; - } - ``` - -### Intervention Mechanism - -When pressure reaches **high** (≥90%): -1. Plugin sends `promptAsync()` message to agent -2. Message includes full visible prompt for review -3. Agent can compress core memory, clear working memory, or continue -4. Intervention tracked in `interventionsSent` counter - -### System Prompt Injection - -``` -[Memory Pressure: 87% (high) - 156,600/180,000 chars] -⚠️ High memory pressure detected. Consider: -- Compressing core_memory blocks (use core_memory_update) -- Clearing resolved errors (use working_memory_clear_slot) -- Removing old pool items (auto-pruned at score < 0.01) -``` - -## Phase 4.5: Storage Governance - -### Purpose -Prevent `.opencode/` directory bloat from accumulating tool output caches and orphaned memory files. - -### Layer 1: Session Deletion Cleanup - -**Trigger**: `experimental.session.deleted` hook - -**Actions**: -1. Remove `.opencode/memory-core/.json` -2. Remove `.opencode/memory-working/.json` -3. Remove `.opencode/memory-working/_pressure.json` -4. Remove `.opencode/memory-working/_compaction.json` - -### Layer 2: Tool Output Cache Sweep - -**Trigger**: Every 500 events (`eventCounter % 500 === 0`) - -**Target**: `.opencode/cache/tool-outputs/` directory - -**Policy**: -- Keep most recent **300 files** (sorted by mtime) -- Delete files older than **7 days** (TTL policy) - -**Logging**: Write sweep results to `.opencode/memory-working/_sweep.json` +### Negative Memory Filtering ```typescript -{ - sessionID: string; - timestamp: string; - eventCounter: number; - results: { - filesScanned: number; - filesDeleted: number; - bytesReclaimed: number; - errors: Array; - }; -} +// Correctly interpreted +"don't remember this" → NOT added to memory +"不要記住這個" → NOT added to memory +"remember this" → added to memory candidates +``` + +### Canonical Deduplication + +```typescript +// Same memory (after normalization) +"Use npm cache for plugins" +"USE NPM CACHE for plugins!!" +"use npm cache for plugins." + +// All map to same canonical key +canonical("Use npm cache for plugins") === "use npm cache for plugins" +``` + +### Compaction Quality Gate + +```typescript +// Rejected (not valuable as long-term memory) +"4832b38 fix: something" // git hash +"Error: something failed" // raw error +"at Object.method (file.ts:42)" // stack trace +"/Users/x/project/file.ts /Users/x/project/other.ts" // path-heavy + +// Accepted +"[decision] Use npm cache for plugin loading" // good pattern +``` + +## File System Layout + +``` +~/.local/share/opencode-working-memory/ +└── workspaces/ + └── {workspaceKey}/ + ├── workspace-memory.json # Long-term memory + └── sessions/ + └── {hashedSessionID}.json # Session state +``` + +### Workspace Key + +```typescript +// First 16 chars of SHA-256 hash of workspace root realpath +const workspaceKey = sha256(realpath(workspaceRoot)).slice(0, 16) ``` ## Performance Considerations ### Memory Budgets -- **Core Memory**: 4,500 chars (injected every message) -- **Working Memory**: ~1,600 chars (injected every message) -- **Total Overhead**: ~6,100 chars per message -### Compaction Behavior -When OpenCode compacts conversation (clears old messages): -- Core memory: **Preserved** (persistent across compactions) -- Working memory: **Preserved** (session-scoped, cleared on session end) -- Pressure state: **Preserved** (tracks across compaction) -- Compaction log: Saved to `_compaction.json` +| Layer | Max Chars | Max Entries | +|-------|-----------|-------------| +| Workspace Memory | 5200 | 28 | +| Hot Session State | 1200 | 8 files, 3 errors | + +### Injection Overhead + +- Workspace memory: ~200-500 chars per message +- Hot session state: ~200-400 chars per message +- Total: ~400-900 chars per message (minimal) ### Storage Footprint -- Each session: 4 JSON files (~5-20 KB total) -- Tool output cache: Max 300 files (~10-50 MB depending on outputs) -- Sweep every 500 events keeps storage bounded + +- Workspace memory: ~2-5 KB per workspace +- Session state: ~1-3 KB per session +- Auto-cleanup on workspace/session deletion ## Extension Points -### Custom Slot Types -To add new slot types: -1. Update `SlotType` union in types -2. Add to `SLOT_CONFIG` with max items -3. Update `formatWorkingMemoryForPrompt()` for display -4. Update extraction heuristics in `tool.execute.after` +### Custom Memory Types -### Custom Pruning Rules -To add pruning heuristics: -1. Update `shouldPrune()` with new detection logic -2. Add to `pruneToolOutput()` with filtering rules -3. Test with representative tool outputs - -### Custom Pressure Thresholds -Adjust in constants: +Add new types in `src/types.ts`: ```typescript -const PRESSURE_THRESHOLDS = { - moderate: 70, - high: 85, - critical: 95, -}; +export type LongTermType = "feedback" | "project" | "decision" | "reference" | "custom"; ``` -## Migration & Compatibility +### Custom Error Categories -### Old Format → New Format -Plugin automatically migrates from old format: +Add new categories in `src/types.ts`: ```typescript -// Old format (pre-Phase 3) -{ items: Array } - -// New format (Phase 3+) -{ slots: Record>, pool: Array } +export type ErrorCategory = "typecheck" | "test" | "lint" | "build" | "runtime" | "custom"; ``` -Migration happens on first load of old format files. +### Custom Extraction Patterns -## File System Layout +Modify `src/extractors.ts` to add new extraction patterns. -``` -.opencode/ -├── memory-core/ -│ └── .json # Core memory blocks -├── memory-working/ -│ ├── .json # Working memory (slots + pool) -│ ├── _pressure.json # Pressure monitoring state -│ ├── _compaction.json # Compaction event log -│ └── _sweep.json # Storage sweep log -└── cache/ - └── tool-outputs/ - └── *.json # Tool output cache (auto-swept) -``` +## Migration Notes -## Security Considerations +### Memory V1 to V2 -- All files written with `0644` permissions (owner read/write, group/others read) -- Directories created with `0755` permissions (owner rwx, group/others rx) -- No sensitive data should be stored in memory blocks (user responsibility) -- Session IDs are opaque identifiers, not derived from sensitive data +The plugin automatically migrates old format files to the new three-layer architecture. No manual intervention needed. --- -**Last Updated**: February 2026 -**Implementation**: `index.ts` (1700+ lines) +**Last Updated**: April 2026 +**Implementation**: `src/plugin.ts`, `src/extractors.ts`, `src/workspace-memory.ts`, `src/session-state.ts` \ No newline at end of file diff --git a/docs/configuration.md b/docs/configuration.md index 7f97c98..3472baa 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -2,250 +2,120 @@ ## Overview -The Working Memory Plugin works out-of-the-box with sensible defaults. Advanced users can customize behavior by modifying constants in `index.ts`. +The Working Memory Plugin works out-of-the-box with sensible defaults. Configuration is defined in `src/types.ts` as constants. -## Core Memory Limits +## Workspace Memory Limits ```typescript -const CORE_MEMORY_LIMITS = { - goal: 1000, // ONE specific task (not project-wide goals) - progress: 2000, // Checklist format (✅ done, ⏳ in-progress, ❌ blocked) - context: 1500, // Current working files + key patterns +const LONG_TERM_LIMITS = { + maxRenderedChars: 5200, // Maximum characters in system prompt + targetRenderedChars: 4200, // Target characters (leave buffer) + maxEntries: 28, // Maximum number of entries + maxEntryTextChars: 260, // Maximum characters per entry text + maxRationaleChars: 180, // Maximum characters per entry rationale }; ``` **Recommendations**: -- Keep **goal** focused on current task (clear when completed) -- Use **progress** for checklists (avoid line numbers, commit hashes, API signatures) -- Use **context** for files you're actively editing (avoid type definitions, function signatures) +- Keep `maxRenderedChars` under 5500 to avoid context bloat +- `maxEntries` of 28 provides good coverage without overwhelming +- Entry text limits ensure entries stay concise -## Working Memory Configuration - -### Slot Limits +## Hot Session State Limits ```typescript -const SLOT_CONFIG: Record = { - error: 3, // Recent errors needing fixes - decision: 5, // Important decisions made - todo: 3, // Current task checklist - dependency: 3, // File/package dependencies +const HOT_STATE_LIMITS = { + maxRenderedChars: 1200, // Maximum characters in system prompt + maxActiveFilesStored: 20, // Maximum files tracked in state + maxActiveFilesRendered: 8, // Maximum files shown in prompt + maxOpenErrorsStored: 5, // Maximum errors tracked + maxOpenErrorsRendered: 3, // Maximum errors shown in prompt + maxRecentDecisionsStored: 8, // Maximum decisions tracked }; ``` -**Tuning**: -- Increase slot limits if you need more items tracked -- Decrease for stricter memory budgets -- Total overhead: ~100-200 chars per item +**Recommendations**: +- Keep `maxRenderedChars` under 1500 for fast prompts +- `maxActiveFilesRendered` of 8 provides good context coverage +- `maxOpenErrorsRendered` of 3 avoids overwhelming error lists -### Memory Pool Decay +## Memory Types -```typescript -const POOL_DECAY_GAMMA = 0.85; // Exponential decay rate (15% per event) -const POOL_MIN_SCORE = 0.01; // Items below this score are pruned +### Long-Term Memory Types + +| Type | Purpose | Stale After (days) | +|------|---------|---------------------| +| `feedback` | User preferences for workspace | 90 | +| `project` | Project-level information | 60 | +| `decision` | Important decisions | 45 | +| `reference` | Key references | 90 | + +### Memory Sources + +| Source | Confidence | Description | +|--------|------------|-------------| +| `explicit` | 1.0 | User explicitly said "remember this" | +| `compaction` | 0.75 | Extracted during conversation compaction | +| `manual` | varies | Added programmatically | + +## Active File Scoring + +Files are ranked by action type: + +| Action | Weight | Description | +|--------|--------|-------------| +| `write` | 4 | File created/overwritten | +| `edit` | 3 | File modified | +| `read` | 2 | File read | +| `grep` | 1 | Grep searched in file | + +Score formula: `count * action_weight * recency_decay` + +## Error Categories + +| Category | Recognition Pattern | +|----------|---------------------| +| `typecheck` | TS errors, TypeScript failures | +| `test` | Test failures | +| `lint` | ESLint warnings/errors | +| `build` | Build failures | +| `runtime` | Uncaught errors, Node exceptions | +| `tool` | Tool execution failures | + +## Storage Paths + +``` +~/.local/share/opencode-working-memory/ +└── workspaces/ + └── {workspaceKey}/ + ├── workspace-memory.json # Long-term memory + └── sessions/ + └── {sessionID}.json # Session state ``` -**Formula**: `score = exp(-γ * age) + mentionCount` - -**Tuning**: -- Lower `γ` (e.g., 0.75) → faster decay, more aggressive pruning -- Higher `γ` (e.g., 0.90) → slower decay, items stay longer -- Lower `POOL_MIN_SCORE` (e.g., 0.005) → more items retained - -### Pool Size Limits +### Workspace Key ```typescript -const POOL_MAX_ITEMS = 50; // Hard limit on pool size +// First 16 characters of SHA-256 hash +const workspaceKey = sha256(realpath(workspaceRoot)).slice(0, 16); ``` -**Tuning**: -- Increase for longer sessions with more context -- Decrease for stricter memory budgets -- Each item adds ~50-150 chars to system prompt +## Customization -## Pressure Monitoring - -### Thresholds +To customize limits, edit the constants in `src/types.ts`: ```typescript -const PRESSURE_THRESHOLDS = { - moderate: 75, // Warning appears in system prompt - high: 90, // Aggressive pruning activates + intervention sent +// Example: Increase workspace memory limit +export const LONG_TERM_LIMITS = { + maxRenderedChars: 6000, // Increased from 5200 + maxEntries: 35, // Increased from 28 + // ... }; ``` -**Tuning**: -- Increase thresholds for more relaxed monitoring -- Decrease for earlier warnings and interventions - -### Context Limit Estimate - -```typescript -const ESTIMATED_CONTEXT_LIMIT = 180000; // Conservative estimate (chars) -``` - -**Note**: OpenCode actual limit varies by model. Adjust based on your observations. - -## Smart Pruning - -### Line Thresholds - -```typescript -// Normal mode (pressure < 75%) -const PRUNE_THRESHOLD_NORMAL = 50; - -// Aggressive mode (75% ≤ pressure < 90%) -const PRUNE_THRESHOLD_AGGRESSIVE = 30; - -// Hyper-aggressive mode (pressure ≥ 90%) -const PRUNE_THRESHOLD_HYPER = 15; -``` - -**Tuning**: -- Increase thresholds to keep more tool output -- Decrease for more aggressive pruning - -### Keep Lines - -```typescript -// Normal mode -const KEEP_LINES_NORMAL = 30; // Keep first/last 30 lines - -// Aggressive mode -const KEEP_LINES_AGGRESSIVE = 20; // Keep first/last 20 lines - -// Hyper-aggressive mode -const KEEP_LINES_HYPER = 10; // Keep first/last 10 lines -``` - -**Tuning**: -- Increase to preserve more context from tool outputs -- Decrease for stricter truncation - -## Storage Governance - -### Session Cleanup - -Automatically triggered on `experimental.session.deleted` hook. No configuration needed. - -### Tool Output Cache Sweep - -```typescript -const SWEEP_INTERVAL = 500; // Trigger every N events -const SWEEP_MAX_FILES = 300; // Keep most recent N files -const SWEEP_TTL_DAYS = 7; // Delete files older than N days -``` - -**Tuning**: -- Increase `SWEEP_INTERVAL` for less frequent sweeps (lower overhead) -- Increase `SWEEP_MAX_FILES` to cache more tool outputs (more disk usage) -- Increase `SWEEP_TTL_DAYS` to keep older files longer - -## Compaction Behavior - -### Item Preservation - -```typescript -const COMPACTION_KEEP_ITEMS = 10; // Preserve N most recent items on compaction -``` - -**Tuning**: -- Increase to preserve more working memory across compactions -- Decrease for stricter memory reset on compaction - -## System Prompt Injection - -### Core Memory Format - -```typescript -// Injected as: - -... -... -... - -``` - -**Customization**: Modify `formatCoreMemoryForPrompt()` in `index.ts` to change format. - -### Working Memory Format - -```typescript -// Injected as: - -Recent session context (auto-managed, sorted by relevance): - -⚠️ Errors: - - item content - -📁 Key Files: - - file path - -(N items shown, updated: HH:MM:SS AM) - -``` - -**Customization**: Modify `formatWorkingMemoryForPrompt()` in `index.ts` to change: -- Section emoji/icons -- Display format -- Item ordering - -### Pressure Warning Format - -```typescript -// Injected as: -[Memory Pressure: 87% (high) - 156,600/180,000 chars] -⚠️ High memory pressure detected. Consider: -- Action item 1 -- Action item 2 -``` - -**Customization**: Modify `formatPressureWarning()` in `index.ts`. - -## Auto-Extraction Heuristics - -### File Path Detection - -```typescript -// Detects: -- Absolute paths: /users/name/project/file.ts -- Relative paths: src/components/Button.tsx -- Dot paths: ./utils/helpers.ts -- Tilde paths: ~/project/file.ts -``` - -**Customization**: Modify regex in `extractFilePaths()`. - -### Error Detection - -```typescript -// Detects: -- "Error:", "ERROR:", "error:" -- Stack traces with "at " prefix -- TypeScript errors with "TS####:" -``` - -**Customization**: Modify `extractErrors()` heuristics. - -### Decision Detection - -```typescript -// Detects: -- "decided to...", "decision:", "chose to..." -- "using X instead of Y" -- "will use X approach" -``` - -**Customization**: Modify `extractDecisions()` heuristics. - -## Environment Variables - -Currently, the plugin does not support environment variables. All configuration is done via constants in `index.ts`. - -**Future Enhancement**: Consider adding `.env` support for: -``` -OPENCODE_WM_CORE_GOAL_LIMIT=1000 -OPENCODE_WM_POOL_DECAY_GAMMA=0.85 -OPENCODE_WM_SWEEP_INTERVAL=500 +**Note**: After customization, rebuild the plugin: +```bash +npm run build ``` ## Performance Tuning @@ -253,123 +123,83 @@ OPENCODE_WM_SWEEP_INTERVAL=500 ### High-Frequency Sessions (500+ messages) ```typescript -// Aggressive pruning -const PRUNE_THRESHOLD_NORMAL = 30; -const PRUNE_THRESHOLD_AGGRESSIVE = 20; - -// Faster decay -const POOL_DECAY_GAMMA = 0.75; - -// More frequent sweeps -const SWEEP_INTERVAL = 250; +// Reduce memory overhead +const HOT_STATE_LIMITS = { + maxRenderedChars: 800, // Reduced + maxActiveFilesRendered: 5, // Reduced + maxOpenErrorsRendered: 2, // Reduced +}; ``` ### Long-Running Sessions (Multi-day) ```typescript // Preserve more context -const POOL_MAX_ITEMS = 100; -const COMPACTION_KEEP_ITEMS = 20; - -// Slower decay -const POOL_DECAY_GAMMA = 0.90; - -// Longer TTL -const SWEEP_TTL_DAYS = 14; +const LONG_TERM_LIMITS = { + maxEntries: 40, // Increased + targetRenderedChars: 5000, // Increased +}; ``` ### Memory-Constrained Environments ```typescript // Strict limits -const CORE_MEMORY_LIMITS = { - goal: 500, - progress: 1000, - context: 800, +const LONG_TERM_LIMITS = { + maxRenderedChars: 3000, + maxEntries: 15, }; -const POOL_MAX_ITEMS = 20; - -// Aggressive pruning -const PRUNE_THRESHOLD_NORMAL = 20; +const HOT_STATE_LIMITS = { + maxRenderedChars: 600, + maxActiveFilesRendered: 4, +}; ``` -## Debugging Configuration - -### Enable Verbose Logging - -Add `console.log()` statements in key functions: - -```typescript -// In loadCoreMemory() -console.log("[Core Memory] Loaded:", memory); - -// In applyDecay() -console.log("[Pool Decay] Pruned items:", prunedCount); - -// In sweepToolOutputCache() -console.log("[Sweep] Deleted files:", deletedCount); -``` +## Debugging ### Inspect Memory Files ```bash -# Core memory -cat .opencode/memory-core/.json | jq +# Workspace memory +cat ~/.local/share/opencode-working-memory/workspaces/*/workspace-memory.json | jq -# Working memory -cat .opencode/memory-working/.json | jq - -# Pressure state -cat .opencode/memory-working/_pressure.json | jq - -# Sweep log -cat .opencode/memory-working/_sweep.json | jq +# Session state +cat ~/.local/share/opencode-working-memory/workspaces/*/sessions/*.json | jq ``` -## Migration Notes +### Clear Workspace Memory -### Upgrading from Pre-Phase 3 - -Old format files are automatically migrated: - -```typescript -// Old format -{ items: Array } - -// New format (auto-migrated) -{ slots: { error: [], decision: [], ... }, pool: [...] } +```bash +# Remove workspace memory (start fresh) +rm ~/.local/share/opencode-working-memory/workspaces/*/workspace-memory.json ``` -No manual intervention required. +### Clear Session State -### Upgrading from Phase 3 to Phase 4.5 - -Storage governance is backward compatible. No migration needed. +```bash +# Remove all session states +rm ~/.local/share/opencode-working-memory/workspaces/*/sessions/*.json +``` ## Best Practices -1. **Core Memory Discipline**: - - Clear `goal` immediately after task completion - - Keep `progress` concise (use checklist format) - - Only put actively edited files in `context` +1. **Workspace Memory Hygiene**: + - Let the plugin extract memories automatically + - Use explicit "remember this" for important information + - Don't manually edit memory files unless testing -2. **Working Memory Hygiene**: - - Clear `error` slot after fixing all errors (`working_memory_clear_slot`) - - Let pool decay naturally (avoid manual removal unless necessary) - - Review working memory periodically (use `working_memory_read`) +2. **Session State**: + - Let the plugin track active files automatically + - Errors are cleared when commands succeed + - No manual intervention needed -3. **Pressure Management**: - - Respond to "moderate" warnings proactively - - Compress core memory at "high" pressure - - Clear working memory at "critical" pressure - -4. **Storage Maintenance**: - - Let sweep run automatically (no manual intervention) - - Delete old session files manually if needed - - Monitor `.opencode/` directory size periodically +3. **Memory Extraction**: + - Use `` during compaction + - Follow the pattern: `- [type] text` + - Quality gate rejects invalid candidates --- -**Last Updated**: February 2026 -**Configuration File**: `index.ts` (constants section) +**Last Updated**: April 2026 +**Configuration File**: `src/types.ts` \ No newline at end of file diff --git a/docs/installation.md b/docs/installation.md index f03ecc5..f02b57a 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -10,7 +10,7 @@ Add to your `~/.config/opencode/opencode.json`: } ``` -Restart OpenCode. The plugin is downloaded and installed automatically — no `npm install` needed. +Restart OpenCode. The plugin activates automatically — no manual setup needed. > **Note**: The correct key is `plugin` (singular), not `plugins`. @@ -22,19 +22,44 @@ Restart OpenCode. The plugin is downloaded and installed automatically — no `n ## Verification -After restarting OpenCode, ask your agent: +After restarting OpenCode, memory context appears automatically in system prompts. You'll see: ``` -Use core_memory_read to show me what you remember + +- [decision] ... (if any long-term memories exist) + + +--- + +- [project] ... (candidates for long-term memory) + + +Active Files: +- path/to/file.ts (action, count) + +Open Errors: (none, or listed) ``` -If the tool responds, the plugin is active. +**No tools to call**. The plugin works automatically via hooks. + +## How Memory Works + +### Workspace Memory (Long-term) + +Persists across sessions. Automatically extracted during compaction when you say "remember this" or when important decisions are made. + +### Hot Session State (Short-term) + +Tracks current session: +- Active files (what you're working on) +- Open errors (unresolved issues) +- Recent decisions (for compaction candidate promotion) ## Troubleshooting ### Plugin Not Loading -**Symptom**: No `core_memory_update` tool available +**Symptom**: No memory context in system prompt **Solution**: 1. Check `~/.config/opencode/opencode.json` uses `"plugin"` (not `"plugins"`) @@ -43,11 +68,21 @@ If the tool responds, the plugin is active. ### Memory Files Not Created -**Symptom**: No `.opencode/memory-core/` or `.opencode/memory-working/` directories +**Symptom**: No `~/.local/share/opencode-working-memory/` directory **Solution**: -1. Ensure OpenCode has write permissions in project directory -2. Trigger memory operations (e.g., use `core_memory_update` tool) +1. Ensure OpenCode has write permissions in home directory +2. Trigger memory operations by working normally (plugin creates files on-demand) +3. Check that plugin is listed in config + +### Memory Not Persisting + +**Symptom**: Workspace memory empty after restart + +**Solution**: +1. Verify you're in the same workspace (different workspace = different memory) +2. Ensure `` were captured during compaction +3. Check `workspace-memory.json` exists ### Type Errors During Development @@ -56,16 +91,45 @@ If the tool responds, the plugin is active. **Solution**: 1. Run `npm install` to install dev dependencies 2. Run `npm run typecheck` to check for errors -3. See [AGENTS.md](../AGENTS.md) for code style guidelines +3. Run `npm test` to verify functionality ## Uninstallation Remove `"opencode-working-memory"` from the `plugin` array in `~/.config/opencode/opencode.json`. -Memory files in `.opencode/memory-*` will persist unless manually deleted. +Memory files in `~/.local/share/opencode-working-memory/` persist unless manually deleted. + +## Manual Memory Management + +### View Workspace Memory + +```bash +cat ~/.local/share/opencode-working-memory/workspaces/*/workspace-memory.json | jq +``` + +### View Session State + +```bash +cat ~/.local/share/opencode-working-memory/workspaces/*/sessions/*.json | jq +``` + +### Clear Workspace Memory + +```bash +rm ~/.local/share/opencode-working-memory/workspaces/*/workspace-memory.json +``` + +### Clear All Session States + +```bash +rm -rf ~/.local/share/opencode-working-memory/workspaces/*/sessions/*.json +``` ## Next Steps -- Read [Architecture Documentation](./architecture.md) to understand how memory tiers work +- Read [Architecture Documentation](./architecture.md) to understand how the three layers work - See [Configuration Guide](./configuration.md) for customization options -- Check [AGENTS.md](../AGENTS.md) for development guidelines + +--- + +**Last Updated**: April 2026 \ No newline at end of file