mirror of
https://github.com/giancarloerra/socraticode.git
synced 2026-07-03 14:05:21 +02:00
Giancarlo Erra
4e41b4604e
Closes the four reviewer-flagged gaps from the previous round:
1. **Phase F wired into the watcher / `codebase_update`.**
`rebuildGraph(path, { skipSymbolGraph: true })` now exposes a
file-import-only build mode. `services/indexer.ts` calls it +
`updateChangedFilesSymbolGraph(...)` when meta exists AND ≤ 50 files
changed (`INCREMENTAL_SYMBOL_THRESHOLD`); falls back to full rebuild
above that. Measured speedup on a 1000-file synthetic repo: full
rebuild 6.55 s → Phase F single-file update 197 ms (~33×).
2. **Real end-to-end scale test.**
New `tests/integration/symbol-graph-scale.test.ts` generates 1000
synthetic Python files × 20 symbols/file (20k symbols) against a real
Qdrant, asserts (a) full rebuild within budget, (b) cold listSymbols /
getImpactRadius queries within budget, (c) Phase F update ≥ 4× faster
than full rebuild. `SCALE_LARGE=1` pushes to 10k files / 200k symbols.
3. **Smoke benchmark numbers captured.**
New `scripts/benchmark-graph.ts` runs `rebuildGraph` against any
target dir and emits JSON + a Markdown row. Numbers for SocratiCode
itself (82 files / 571 symbols / 9914 call edges / 0.90 s / 167 MB
RSS) and the synthetic 1000-file repo are now in DEVELOPER.md
§ "Real-world benchmark numbers".
4. **Logger test flake fixed.**
`services/logger.ts` exposes `setLogLevel` / `getLogLevel`;
`tests/unit/logger.test.ts` pins the level in beforeEach and restores
in afterEach. Verified deterministic with `SOCRATICODE_LOG_LEVEL=debug`
set in the shell environment.
### Bug discovered + fixed by the new benchmark
Running `scripts/benchmark-graph.ts` against SocratiCode itself crashed
the symbol graph build with `TypeError: existing.push is not a function`.
Root cause: shard maps used `shard[name]` bracket access on a plain
`{}`, which returned `Object.prototype.constructor` (a function) for
common method names like `constructor`, `toString`, `hasOwnProperty`.
Fixed by guarding all reads with `Object.hasOwn` in
`services/code-graph.ts` and `services/symbol-graph-incremental.ts`.
Added a regression test in
`tests/integration/symbol-graph-incremental.test.ts`.
### QA
- Biome lint: clean (auto-fixed 1 file).
- VS Code Problems panel: clean.
- Unit tests: 676/676 pass (29 files); reproducible.
- Integration tests touched: 45/45 pass (incremental, scale,
indexer, code-graph).
- CodeRabbit review: no findings.
- Snyk Code: 0 issues.
### Doc updates
- DEVELOPER.md: removed "watcher still triggers full rebuild" wording,
added "Real-world benchmark numbers" subsection with measured table.
- CHANGELOG.md: removed "Known Limitations" block; added new
Bug Fixes entries (prototype keys, logger flake) and a Performance
entry for the wired Phase F path with measured numbers.
67 lines
2.4 KiB
TypeScript
67 lines
2.4 KiB
TypeScript
// SPDX-License-Identifier: AGPL-3.0-only
|
|
// Copyright (C) 2026 Giancarlo Erra - Altaire Limited
|
|
|
|
/**
|
|
* scripts/benchmark-graph.ts
|
|
*
|
|
* Smoke benchmark: builds the code graph + symbol graph for a target
|
|
* repository and prints timing / count / memory results as JSON.
|
|
*
|
|
* Usage:
|
|
* npx tsx scripts/benchmark-graph.ts <absolute-path>
|
|
* npx tsx scripts/benchmark-graph.ts # defaults to cwd
|
|
*
|
|
* The script also prints a short Markdown line suitable for pasting into
|
|
* DEVELOPER.md's "Real-world benchmark numbers" table.
|
|
*/
|
|
|
|
import path from "node:path";
|
|
import process from "node:process";
|
|
import { projectIdFromPath } from "../src/config.js";
|
|
import { rebuildGraph } from "../src/services/code-graph.js";
|
|
import { setLogLevel } from "../src/services/logger.js";
|
|
import { loadSymbolGraphMeta } from "../src/services/symbol-graph-store.js";
|
|
import { waitForQdrant } from "../tests/helpers/setup.js";
|
|
|
|
async function main(): Promise<void> {
|
|
setLogLevel("warn"); // keep stderr quiet for clean JSON output
|
|
const target = path.resolve(process.argv[2] ?? process.cwd());
|
|
await waitForQdrant();
|
|
|
|
const projectId = projectIdFromPath(target);
|
|
const memBefore = process.memoryUsage().heapUsed;
|
|
const start = Date.now();
|
|
const graph = await rebuildGraph(target);
|
|
const elapsedMs = Date.now() - start;
|
|
const memAfter = process.memoryUsage().heapUsed;
|
|
|
|
const meta = await loadSymbolGraphMeta(projectId);
|
|
|
|
const result = {
|
|
target,
|
|
projectId,
|
|
elapsedMs,
|
|
fileCount: graph.nodes.length,
|
|
edgeCount: graph.edges.length,
|
|
symbolCount: meta?.symbolCount ?? null,
|
|
callEdgeCount: meta?.edgeCount ?? null,
|
|
unresolvedPct: meta?.unresolvedPct ?? null,
|
|
heapDeltaMb: Math.round(((memAfter - memBefore) / 1024 / 1024) * 100) / 100,
|
|
rssMb: Math.round((process.memoryUsage().rss / 1024 / 1024) * 100) / 100,
|
|
nodeVersion: process.version,
|
|
timestamp: new Date().toISOString(),
|
|
};
|
|
|
|
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
|
|
|
|
// Also emit a Markdown row.
|
|
const date = result.timestamp.slice(0, 10);
|
|
const md = `| ${date} | \`${path.basename(target)}\` | ${result.fileCount} | ${result.symbolCount ?? "—"} | ${result.callEdgeCount ?? "—"} | ${(elapsedMs / 1000).toFixed(2)} s | ${result.rssMb} MB |`;
|
|
process.stderr.write(`\nMarkdown row:\n${md}\n`);
|
|
}
|
|
|
|
main().catch((err) => {
|
|
process.stderr.write(`benchmark failed: ${err instanceof Error ? err.message : String(err)}\n`);
|
|
process.exitCode = 1;
|
|
});
|