Run JSON tool calls a model writes after a sentence, or several in one reply (#753)

Tool calls a local model writes as fenced JSON after a sentence, or several in one reply, are run when every one names an offered tool; words after the last call, other code or an unclosed fence keep the reply as text.

crates/ai/src/openai.rs

@@ -492,6 +492,52 @@ fn json_action(text: &str) -> Option<ReactStep> {
     input.is_object().then(|| ReactStep::Action { tool: tool.to_string(), input })
 }
 
+/// The calls in a reply that ends in fenced JSON calls after a sentence or between them
+/// ("Let's list the files." then a ```json fence), as small local models write them (#417).
+/// Every fence must be a complete `json` (or unlabelled) call, and nothing but blank lines may
+/// follow the last one; anything else (other code, a `~~~` fence, a fence inside a fence, an
+/// unclosed fence, words after the call, a JSON object quoted inline) means the reply is text,
+/// as before.
+fn fenced_calls(text: &str) -> Option<Vec<(String, Value)>> {
+    let mut calls = Vec::new();
+    let mut lines = text.lines();
+    let mut words_after = false;
+    while let Some(line) = lines.next() {
+        let opening = line.trim();
+        // Anything in a tilde fence is an example, whatever it holds.
+        if opening.starts_with("~~~") {
+            return None;
+        }
+        let length = opening.bytes().take_while(|byte| *byte == b'`').count();
+        if length < 3 {
+            words_after |= !opening.is_empty() && !calls.is_empty();
+            continue;
+        }
+        let closes = |line: &str| {
+            let line = line.trim();
+            line.len() >= length && line.bytes().all(|byte| byte == b'`')
+        };
+        let mut body = String::new();
+        loop {
+            let next = lines.next()?;
+            if closes(next) {
+                break;
+            }
+            body.push_str(next);
+            body.push('\n');
+        }
+        let language = opening.get(length..)?.trim();
+        // The body is the call itself: a fence inside it is a quoted example.
+        if !matches!(language, "" | "json") || body.trim_start().starts_with(['`', '~']) {
+            return None;
+        }
+        let Some(ReactStep::Action { tool, input }) = json_action(&body) else { return None };
+        calls.push((tool, input));
+        words_after = false;
+    }
+    (!calls.is_empty() && !words_after).then_some(calls)
+}
+
 /// The first balanced `{…}` in `s`, parsed.
 fn extract_json_object(s: &str) -> Option<Value> {
     let start = s.find('{')?;
@@ -533,6 +579,22 @@ fn react_postprocess(r: &mut TurnResult, tools: &[ToolDef], react: bool) {
     let text = r.message.text();
     let json_call = json_action(&text);
     let from_json = json_call.is_some();
+    // Calls after a sentence run only when every one names a tool the model was offered, and
+    // never in a reply that uses the text protocol's own markers (a final answer that ends in an
+    // example is still the answer).
+    let offered = |tool: &str| tools.iter().any(|definition| definition.name == tool);
+    let protocol = ["Final Answer:", "Action:"].into_iter().any(|marker| text.contains(marker));
+    if !from_json
+        && !protocol
+        && let Some(calls) = fenced_calls(&text).filter(|calls| calls.iter().all(|(tool, _)| offered(tool)))
+    {
+        for (tool, input) in calls {
+            let n = REACT_COUNTER.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
+            r.message.blocks.push(Block::ToolUse { id: format!("react_{n}"), name: tool, input });
+        }
+        r.stop = StopReason::ToolUse;
+        return;
+    }
     let step = json_call.or_else(|| react.then(|| parse_react(&text)));
     if let Some(ReactStep::Action { tool, input }) = step
         && (!from_json || tools.iter().any(|definition| definition.name == tool))
@@ -550,6 +612,56 @@ mod tests {
     use super::*;
     use crate::model::Effort;
 
+    #[test]
+    fn calls_after_a_sentence_are_read_and_examples_are_not() {
+        let call = |tool: &str| format!("```json\n{{\"name\":\"{tool}\",\"arguments\":{{\"path\":\"app\"}}}}\n```");
+        let names = |text: &str| fenced_calls(text).map(|calls| calls.into_iter().map(|(tool, _)| tool).collect::<Vec<_>>());
+        // What qwen2.5-coder:14b wrote in a live build.
+        let after = format!("It looks like the file is missing. Let's list the files.\n\n{}", call("look"));
+        assert_eq!(names(&after), Some(vec!["look".to_string()]));
+        let several = format!("{}\n\nThen:\n{}\n\n", call("look"), call("write"));
+        assert_eq!(names(&several), Some(vec!["look".to_string(), "write".to_string()]));
+        assert_eq!(names("```\n{\"name\":\"look\",\"parameters\":{}}\n````"), Some(vec!["look".to_string()]), "unlabelled, longer close");
+        for text in [
+            format!("{}\nThis is an example.", call("look")),
+            format!("Here's Rust:\n```rust\nfn main() {{}}\n```\n{}", call("look")),
+            format!("{}\n```json\n{{\"name\":\"look\",\"arguments\":{{}}}}", call("look")),
+            "For example, use {\"name\":\"look\",\"arguments\":{}}.".to_string(),
+            "```json\n{\"name\":\"look\",\"arguments\":{},\"description\":\"x\"}\n```".to_string(),
+            "```json\n[{\"name\":\"look\",\"arguments\":{}}]\n```".to_string(),
+            "No calls here.".to_string(),
+            // An example in a tilde fence, then a real-looking call.
+            format!("~~~markdown\n{}\n~~~\n{}", call("write"), call("look")),
+            // A four-backtick fence quoting a three-backtick call.
+            format!("Like this:\n````\n{}\n````", call("write")),
+        ] {
+            assert_eq!(names(&text), None, "{text}");
+        }
+
+        // Only tools that were offered run, and then all of them.
+        let tool = |name: &str| ToolDef { name: name.into(), description: String::new(), schema: json!({"type": "object"}) };
+        let turn = |text: &str| TurnResult {
+            message: Msg { role: Speaker::Assistant, blocks: vec![Block::Text { text: text.into() }] },
+            stop: StopReason::EndTurn,
+            usage: Usage::default(),
+            model: String::new(),
+        };
+        let mut r = turn(&several);
+        react_postprocess(&mut r, &[tool("look"), tool("write")], false);
+        assert_eq!(r.stop, StopReason::ToolUse);
+        assert_eq!(r.message.tool_uses().into_iter().map(|(_, name, _)| name.to_string()).collect::<Vec<_>>(), ["look", "write"]);
+        let mut r = turn(&several);
+        react_postprocess(&mut r, &[tool("look")], false);
+        assert!(r.message.tool_uses().is_empty() && r.stop == StopReason::EndTurn, "one isn't offered: none run");
+        // A final answer that ends in an example stays the answer.
+        let answer = format!("Final Answer: it's built. To list the files again:\n{}", call("look"));
+        for react in [false, true] {
+            let mut r = turn(&answer);
+            react_postprocess(&mut r, &[tool("look")], react);
+            assert!(r.message.tool_uses().is_empty() && r.stop == StopReason::EndTurn, "react={react}");
+        }
+    }
+
     /// Only the real OpenCode Go base gets its session header: not a look-alike host, path or scheme.
     #[test]
     fn opencode_go_base_is_recognised_exactly() {

docs/ROADMAP.md

@@ -230,19 +230,24 @@ been run live.
       instead of 33-41 (23,000-30,000): the Build tools and the Core tools the brief names, not the
       study set a model explores with instead of writing (#417, `LOCAL_BUILDER_CORE`). Measured as
       text, not on a model; Ollama's default context (4k) was too small for the old list alone
-- [ ] Run #417's case live (Ollama, `qwen2.5-coder:14b`, "a scientific calculator") before and
+- [x] Run #417's case live (Ollama, `qwen2.5-coder:14b`, "a scientific calculator") before and
       after this: does it call `build_file_write` now? If it still doesn't, the model can't build.
-      Run so far: with the lean list it calls `build_checklist`, `build_command_run` and
-      `build_file_edit` (as JSON text, which #554 runs), but not `build_file_write` yet; the two
-      lines below stop it first
+      Yes, with the three fixes below: it wrote and edited `app/Cargo.toml` and `app/src/main.rs`
+      over about 50 calls, fixed two compile errors the checks reported, and check 3 said "Build
+      succeeded". It stopped at the round limit without the `--screenshot` option (#753)
 - [x] The system prompt told every model to call `core_brief_write` and `core_plan_update` first and
       `core_handoff_write` last, tools a lean local builder isn't offered, so the model asked only
       for `core_brief_write` and did nothing else for three checks. A lean builder's prompt leaves
       the memory steps out; live, the same model then used the Build tools
-- [ ] A call written after a sentence ("Let's list the files.\n```json {…}```") isn't run: #554
+- [x] A call written after a sentence ("Let's list the files.\n```json {…}```") isn't run: #554
       reads a reply that is only the JSON. Half the model's calls came this way and the turn ended
-      with nothing done. Run a reply that ends in one fenced call to an offered tool, and say in
-      the transcript that it was read from text
+      with nothing done. A reply that ends in complete `json` fences, each a call to an offered
+      tool, with only words before or between them, now runs them all (`fenced_calls`, ids
+      `react_N` as other text calls); words after the last fence, other code, an unclosed fence or
+      a tool that wasn't offered keep it text (#753)
+- [ ] A local builder doesn't add the `--screenshot` option the check needs before the round limit:
+      run 4 compiled at check 3, its last. See whether more rounds get there, or whether a small
+      model needs the option written for it (a template in the Rust steps)
 - [x] The model ran `cargo new app` in `app/` (commands run there), so the project landed in
       `app/app/` and every check said "No project to build found in app/ yet". The check now says
       where a cargo or npm project one folder down is, and the Rust steps say `app/Cargo.toml`,

docs/STATUS.md

@@ -1,7 +1,7 @@
 # Status
 
 What works today and how it was checked. The plan, owners and order of work are in
-[ROADMAP.md](ROADMAP.md). Last updated 2026-11-02. The baseline audit on 2026-10-23 from `main` had 923 tests passing;
+[ROADMAP.md](ROADMAP.md). Last updated 2026-11-04. The baseline audit on 2026-10-23 from `main` had 923 tests passing;
 workspace clippy, layers, fmt and the secret scan of all history were clean. Chinese UI checks are listed separately below.
 
 ## What works, and how it was checked
@@ -30,7 +30,7 @@ workspace clippy, layers, fmt and the secret scan of all history were clean. Chi
 | Electron window comparisons | `app_graphs_compare` retains every unassigned window and every construction with a reused local binding. Windows are paired by module, binding and occurrence, so inserting comments does not hide preference changes. Colliding display labels receive a separate `#N` suffix without changing literal binding names or occurrence indices | Synthetic applications parsed through the ordinary analyzer exercise anonymous and reused bindings, shifted source lines, unequal counts, literal names resembling occurrence or collision suffixes, and stable labels when only one window changes. Pairing remains positional within each binding group; reordered constructions and real application releases were not checked for this correction |
 | Rebuild workflow (Study → Build → Check → Refine) | `crates/ai/src/rebuild/` (kinds, checks and stacks are pluggable: docs/extending.md). Three kinds: website, app, game (a project whose stack is a game engine is always a game). The request decides how to build: a language, framework, engine or platform it names wins over the kind's default stack (Rust + egui for apps, Unity for games, the original's for websites), and the build check, the recorded stack and Play follow what was built (an Electron app opens in its own Electron; a plain static site has nothing to build and passes the build check as it is). Websites: Study also captures up to 4 other pages of the site; menus are opened on a computer and on a phone; the check serves the build on loopback and compares every page at 1440, 768 and 390 px (likeness = mean of layout and perceptual detail, SSIM, averaged over pages, leaving out the parts of the original that change by themselves, found by photographing it three times; the note adds text, page structure, the colours, font, text sizes and spacing the build misses, and per width a page that scrolls sideways, heading sizes and sections whose padding, items per row or content column differ, the errors, failed requests and accessibility gaps the build adds, and the opened menus and pop-ups). A stack or library named in the request (Next.js, shadcn/ui, Tailwind CSS…, `stacks.toml`) is what the builder uses; with none named, a website's stack is the builder's choice (told to match what the original is built with) and the stack it used is recorded from app/package.json. The builder leads: it studies, asks the tools follow-up questions, records what it can't find out as unknowns (`core_unknown_add`; the run reports the open ones), and tests its own build mid-turn with `rebuild_check` (the same checks, for every kind); a check with anything it couldn't compare, or any claim that fails (a website page that scrolls sideways, logs new errors, fails to load a file, lacks a landmark, or a menu that doesn't open; for games each self-test step, for Deep-mode sheets each original number in the code, for console apps the same exit and output as the original's runs), isn't good enough; apps and games by whether they build. Up to 6 refinement rounds, or until a website reaches 90% likeness. A turn cut off by a time limit, crash or dropped connection resumes (up to 3 times); builder turns may run 24 h. A project copied from another computer (or moved to another folder) keeps building with Claude Code: the project keeps a copy of the builder's session (`agent-session.jsonl`) after every turn and puts it back where Claude Code finds it on a computer without it, so the same conversation carries on (#451); when the session still isn't there, a new one starts from the brief, what's in app/ and the kept requests and replies (or the last summary), and questions about the build put the kept session back too, else are answered from the files and forget the missing session (#452). Live runs with Claude Code 2.1.316 on one Mac (2026-11-02), a second computer stood in for by a separate signed-in `CLAUDE_CONFIG_DIR`: a copied project's session was put back and resumed (the builder recalled its first reply from memory); a kept copy of another session was refused and the new session used the handoff; a real project synced from Windows whose session was missing started afresh and reached Ready. Not yet run between two real computers | Unit tests for prompts, stack choice, site detection, feed folding; live runs reached Build and decompiling (below) |
 | Context-summary usage | Successful API context summaries contribute to the message totals consumed by build usage and receive a `model_turn` journal entry with `purpose: context_summary`. Cached-token pricing is retained; an unknown summary price makes the total unknown, and a failed call adds no invented usage | Scripted-model tests exercise the real `ApiConversation`, transcript resumption, exact input/output and cache-price totals, journal entries, empty summaries, local zero cost, unknown prices and failed summaries. No live provider or end-to-end rebuild was used for this correction |
-| Local model builds | Not yet: Ollama 0.40.7 with `qwen2.5-coder:14b` (32k context) studies, plans and runs Build tools, written as JSON text, but its project lands one folder down (`app/app/`) and calls after a sentence aren't run, so it stops at "No project to build found in app/ yet" (R5.4). A lean local builder's prompt no longer asks for memory tools it isn't offered | Two live CLI builds of "a scientific calculator" on Windows (RTX 5090), 2026-11-04 |
+| Local model builds | Compiles, not yet Ready: Ollama 0.40.7 with `qwen2.5-coder:14b` (32k context) writes its calls as JSON text, which run whole or after a sentence; it builds an eframe app that compiles by check 3, then stops at the round limit without the `--screenshot` option the check needs (R5.4). A lean local builder's prompt asks for no memory tools it isn't offered, and a project one folder down is named in the check | Four live CLI builds of "a scientific calculator" on Windows (RTX 5090), 2026-11-04, one fix added per run |
 | Backend context budgets | `context_tokens` in `[[ai.backends]]` controls when an API conversation is trimmed and summarised. Zero or omission retains local/cloud defaults; a nonzero conversation override takes priority. Before each attempt, including a fallback, fitting adds estimated history growth since the last successful request to its reported input size. Trimming resets that reference, and successful requests refresh it; summary usage remains in message totals | Regressions cover settings and model construction, defaults/overrides, smaller fallback budgets, large-budget arithmetic and new tool output growing beyond a fallback budget even when the history estimate alone is below the reported input. The latest tool results are preserved, and a short follow-up does not count already-sent history twice. This does not set server context parameters or guarantee that a complete request fits; no live provider was used |
 | Games | Engine detection for 12 engines (Unreal, Unity, Godot, Source, GameMaker, RPG Maker, Ren'Py, MonoGame, LÖVE, CryEngine, Cocos, Cube) with versions. Unreal and Godot games are rebuilt in the same engine, others in Unity; installed engines are found, Godot installs itself, others get guided installs. Picking an inner folder (Engine, Binaries, Paks…) finds the game's folder. Just look inside a game shows what it's made of (`core_target_triage` on its folder: engine and version, how its code ships, the libraries its Unity assembly names give away, its own code, scenes and size) and opens it in the chat to ask about it. The builder keeps `app/MECHANICS.md` and its structured twin `app/mechanics.json` (checked after each build), in Deep mode only, may reuse the original's readable files for private use (never decrypting; generated pictures and 3D models replace its art when the person turns them on; Clean room copies none of its files), lists them in `CREDITS.md` (checked, with the private-use line in `README.md`); a game is ready only when its `--self-test` shows the player moving, acting and the HUD drawn, and the result can be started from the project page | Unit tests; live runs on Windows |
 | Clean room and Deep mode | Clean room (the default) runs only the tools in `engine/src/clean_room.rs`: ones that touch no target or watch from outside (the website as a visitor sees it, pictures of a window, its controls, recorded runs, a phone's screen). Every other tool, a plugin's included, is refused in the app, CLI, MCP and agents, with a way to turn on Deep mode. A Clean room build studies an app or a game only from the person's pictures and words (and a picture of its window when runtime study is on); its builder's reads and commands go through an approval rule that keeps it out of the original's folder and site (with Claude Code, and with ACP agents for what they ask about; other agents aren't held to it, see roadmap R2.2). Deep mode (`deep_mode` in Settings, `--deep` for one CLI run) uses every tool: the builder sees them all, and a website's study also reads its JavaScript and watches its start page for ten seconds (capture/how-it-works.md). Ghidra adds its decompiler in Deep mode when it's installed: it is found, or installed after asking. Study starts one analysis of the main program, saved and shared with the builder's tool server through a lock file (up to 8 h, Java gets half the machine's memory or the amount set in Settings → Engines, 4 to 31 GB, four slow passes skipped). The builder must decompile before the build is checked; addresses the built-in analysis can't place go to Ghidra, which makes a function on demand (for this run only) where its analysis has none, e.g. code reached only through tables of pointers. Without Ghidra, Deep mode uses the built-in engines | Live runs with Ghidra 12.1.4 on Windows (a shared analysis, and a decompile in seconds on a large program). Live runs with Ghidra 12.2 on Linux x86-64 (decompiles by name in position-independent programs and shared libraries, whose addresses Ghidra moves). The guided install of Ghidra and its Java is offered on Linux and macOS too, but has only been tried on Windows. Clean room and Deep mode: engine, AI, MCP, CLI and app tests; no live Clean room build run yet |