mirror of
https://github.com/aaif-goose/goose.git
synced 2026-07-03 14:10:03 +02:00
fix: improve smoke test prompt for reliable tool calling
The previous prompt 'please list files in the current directory' was ambiguous and didn't explicitly require tool usage. Models like qwen/qwen3-coder and z-ai/glm-4.6 would sometimes respond with text describing what they would do instead of actually calling the tool. The new prompt explicitly instructs the model to immediately call the shell tool without asking for confirmation, which should improve reliability for models with weaker tool-calling capabilities.
This commit is contained in:
@@ -85,7 +85,7 @@ for provider_config in "${PROVIDERS[@]}"; do
|
||||
echo "Model: ${MODEL}"
|
||||
echo ""
|
||||
TMPFILE=$(mktemp)
|
||||
(cd "$TESTDIR" && "$SCRIPT_DIR/target/release/goose" run --text "please list files in the current directory" --with-builtin "$BUILTINS" 2>&1) | tee "$TMPFILE"
|
||||
(cd "$TESTDIR" && "$SCRIPT_DIR/target/release/goose" run --text "Immediately call the shell tool to run 'ls -la'. Do not ask for confirmation." --with-builtin "$BUILTINS" 2>&1) | tee "$TMPFILE"
|
||||
echo ""
|
||||
if grep -qE "$SUCCESS_PATTERN" "$TMPFILE"; then
|
||||
echo "✓ SUCCESS: Test passed - $SUCCESS_MSG"
|
||||
|
||||
Reference in New Issue
Block a user