Name the program an agent test ran, and say when a saved agent path is gone (#809)

`ai test` for a coding agent names the program it ran and, when that was not the saved one, the saved path it replaced; `doctor` says a saved path has no program there instead of blaming PATH. Refs #721.

crates/ai/src/lib.rs

@@ -327,6 +327,13 @@ pub struct TestReport {
     /// whether it's signed in, has a model and quota shows only when it gets a message (#48).
     #[serde(default)]
     pub version_only: bool,
+    /// The coding agent's program that ran, which `reply` is the answer of.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub program: Option<String>,
+    /// The saved program that couldn't be used, when `program` ran in its place (#721: an update
+    /// moved it), so a passing test doesn't hide a setting that's out of date.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub saved_program_replaced: Option<String>,
 }
 
 /// Text kept in the journal for a user message (bounded).
@@ -947,6 +954,8 @@ impl AiHub {
                 latency_ms: start.elapsed().as_millis() as u64,
                 reply: out.stdout_text().lines().next().unwrap_or_default().trim().to_string(),
                 version_only: true,
+                saved_program_replaced: e.options.command.as_deref().map(str::trim).filter(|c| !c.is_empty() && *c != program).map(str::to_string),
+                program: Some(program),
             });
         }
         let mut slot = self.build_model(&e)?;
@@ -969,6 +978,8 @@ impl AiHub {
             latency_ms: start.elapsed().as_millis() as u64,
             reply: r.message.text().trim().to_string(),
             version_only: false,
+            program: None,
+            saved_program_replaced: None,
         })
     }
 
@@ -1047,7 +1058,11 @@ fn reach(e: &BackendEntry, key: Option<String>) -> Result<u64, String> {
         }
         BackendKind::AgentCli => {
             let program = cli_agents::program_of(&e.config, &e.options);
-            craft_anything_platform::which(&program).map(|_| ()).ok_or_else(|| format!("{program} isn't on PATH"))
+            // A saved path that's gone (#721): PATH has nothing to do with it.
+            let by_path = std::path::Path::new(&program).parent().is_some_and(|p| !p.as_os_str().is_empty());
+            craft_anything_platform::which(&program)
+                .map(|_| ())
+                .ok_or_else(|| if by_path { format!("there's no program at {program}") } else { format!("{program} isn't on PATH") })
         }
     };
     r.map(|_| start.elapsed().as_millis() as u64)

crates/ai/src/tests.rs

@@ -103,6 +103,30 @@ fn a_coding_agent_test_says_it_only_checked_the_version() {
     let r = h.test_backend("Agent").unwrap();
     assert!(r.version_only && r.reply.starts_with("cargo "), "{r:?}");
     assert_eq!(serde_json::to_value(&r).unwrap()["version_only"], true, "the CLI's ai test prints it");
+    assert_eq!((r.program.as_deref(), r.saved_program_replaced), (Some("cargo"), None), "the saved program ran");
+}
+
+/// #721: a saved path an update removed. A test that passes through the usual program says so, and
+/// doctor doesn't blame PATH when nothing can be found.
+#[test]
+fn a_saved_program_that_is_gone_is_named_by_test_and_doctor() {
+    let (h, _d) = hub();
+    let dir = tempfile::tempdir().unwrap();
+    let gone = dir.path().join("old-build").join("agent").display().to_string();
+    let backend = |name: &str, model: &str| BackendEntry {
+        config: BackendConfig { name: name.into(), kind: BackendKind::AgentCli, model: model.into(), base_url: None, ssh_host: None, enabled: true },
+        options: BackendOptions { command: Some(gone.clone()), ..Default::default() },
+    };
+    // cargo stands in for the agent's usual program: it's there wherever these tests run.
+    h.add_backend_with(backend("Usual", "cargo"), None).unwrap();
+    h.add_backend_with(backend("Nowhere", "no-such-agent-program"), None).unwrap();
+
+    let r = h.test_backend("Usual").unwrap();
+    assert_eq!((r.program.as_deref(), r.saved_program_replaced.as_deref()), (Some("cargo"), Some(gone.as_str())));
+
+    let reach = h.check_reachability();
+    let nowhere = reach.iter().find(|(name, _)| name == "Nowhere").map(|(_, r)| r.clone()).unwrap();
+    assert_eq!(nowhere, Err(format!("there's no program at {gone}")));
 }
 
 /// Every known agent with an adapter of its own is offered when adding a coding agent.

docs/ROADMAP.md

@@ -441,9 +441,11 @@ Owner: open · Status: in progress
 - [x] A relative program path started in another folder is looked up where it was started
       before it's reported as missing (Unix: a script whose interpreter is missing was said to
       have no file there)
-- [ ] When a saved Codex desktop path is gone, offer the CLI found in its newest build folder
-      (asking first, never replacing a custom program silently), and let Detect/Test check the
-      configured program itself (#721)
+- [x] `ai test` names the program it ran and the saved one it replaced, and `doctor` says a saved
+      path has no program there instead of blaming PATH (#721)
+- [ ] Settings says when a saved agent program is gone and which one runs in its place; today a
+      build moves to the newest build folder or the usual program without saying so, and the
+      setting keeps the old path (read on a worker, not while drawing) (#721)
 - [x] GitHub Copilot through its own CLI login and ACP, without credential import:
       discovery, MCP-only tool restrictions, model selection and fresh read-only questions;
       unit tests and CLI 1.0.105 with an isolated home and loopback model, all five access modes

docs/STATUS.md

@@ -15,7 +15,7 @@ workspace clippy, layers, fmt and the secret scan of all history were clean. Chi
 | SQLite added columns | Short records include missing trailing fields using literal column defaults, or NULL when absent; unsupported default expressions produce NULL with a row note. Stored fields and records longer than the schema retain their values. Defaults are filled before restoring rowid aliases | Nine hand-built parser regressions and an engine tool test cover added-column defaults, rewritten records and short records with a rowid alias, including a missing alias field |
 | OpenCode models and thinking | OpenCode V2 locally and over SSH; Settings → AI lists only that instance's enabled chat models and supported Low/Medium/High/Xhigh/Max variants. Selection is validated and saved through UI, CLI and `ai_models_list` / `ai_model_set`. Explicit-selection execution is refused until native MCP integration exists; local ACP runs require clearing the selection. See [OpenCode](opencode.md). | Stand-in catalogue/process, persistence, remote-command and UI tests; local live catalogue checks (34 models, all five depths represented). Regression checks reject native chat/builds before process I/O with ACP on/off, local/SSH and model-only/depth-only/combined selections. No live SSH host or paid generation used |
 | Claude desktop app without Claude Code | When the `claude` command isn't found but the Claude desktop app is (Windows: the Store package's `claude-desktop` app alias or the older installer's `AnthropicClaude` folder; macOS: `Claude.app`), the builder dialog under "No coding agent was found" and `ai detect` (on stderr) say the desktop app is installed but building needs Claude Code, installed apart (#259). Typed as `DetectHint` from `AiHub::detect_hints` | A regression finds the app in temp folders laid out as each Windows installer leaves it, and nothing on Linux; a UI test shows the hint in the dialog. On Windows 10 the real Store package's alias was found. Not checked: macOS, the older installer, and a computer with no `claude` installed |
-| Missing program paths | A program started by its path (a saved agent program, for one) that has no file there is reported with that path, as possibly moved by an update, with reason `path_missing`, instead of "isn't installed or isn't on your PATH"; a bare name still speaks of PATH. A relative path is looked up where the program starts (on Unix, its working folder), so a script there whose interpreter is missing isn't called a path with no file. It stays a missing program (`EngineMissing`), so builder turns aren't retried (#721) | A regression starts a program at a path with no file on Windows 10 and checks the message, remediation and reason, with a bare-name control; on Linux (Docker), a relative script with a missing interpreter started in its folder, through `Process` and `Launch`. Not checked: a real Codex desktop update, or the app's Settings screen with a stale path |
+| Missing program paths | A program started by its path (a saved agent program, for one) that has no file there is reported with that path, as possibly moved by an update, with reason `path_missing`, instead of "isn't installed or isn't on your PATH"; a bare name still speaks of PATH. A relative path is looked up where the program starts (on Unix, its working folder), so a script there whose interpreter is missing isn't called a path with no file. It stays a missing program (`EngineMissing`), so builder turns aren't retried (#721). `ai test` names the agent program it ran and the saved one it replaced (`program`, `saved_program_replaced`), and `doctor` says a saved path has no program there | `a_saved_program_that_is_gone_is_named_by_test_and_doctor` runs both with a removed path. A regression starts a program at a path with no file on Windows 10 and checks the message, remediation and reason, with a bare-name control; on Linux (Docker), a relative script with a missing interpreter started in its folder, through `Process` and `Launch`. Not checked: a real Codex desktop update, or the app's Settings screen with a stale path |
 | Mach-O Objective-C pointers | Chained offsets mapped only into file-empty `__PAGEZERO` reach the image-base fallback; pointer candidates must be backed by file bytes | Synthetic thin and fat Mach-O regression, plain-pointer and authenticated-rebase controls, null and unmapped-pointer checks. Real binaries were not checked for this correction |
 | Apple app graphs | Selecting a macOS bundle recognises its root-relative `Contents/Info.plist` and retains its service/extension type; nested XPC bundles appear in `services` and as `xpc-service` graph nodes, including their executables when hosted in frameworks | Synthetic inventory tests, including parent-directory and flat bundle layouts, selected services/extensions, framework-hosted service executables, and bare-program controls. Ordinary selected directories with `Contents/Info.plist` need an Apple bundle suffix; archives retain layout-based recognition. Framework libraries and dylibs stay excluded from executables. Real app bundles were not checked for this correction |
 | Chinese UI | Simplified (`zh-hans`) and Traditional (`zh-hant`) Chinese in Settings → General → Language; System chooses a supported OS language, otherwise English. Both catalogs cover UI controls and tool titles; translated titles are searchable. Raw tool data, AI text, logs, CLI/MCP and the separate browser viewer keep their original text. See [localisation](localization.md). | Catalog integrity and locale-resolution tests; `cargo xtask i18n-coverage`; isolated UI language-switch test. Platform visual checks are recorded separately. |