diff --git a/RELEASE_CHECKLIST.md b/RELEASE_CHECKLIST.md new file mode 100644 index 0000000000..181521bd09 --- /dev/null +++ b/RELEASE_CHECKLIST.md @@ -0,0 +1,117 @@ +# goose Release Manual Testing Checklist + +## Version: _____ + +Make a copy of this document for each version and check off as steps are verified. + +## Provider Testing + +- [ ] Run `./scripts/test_providers.sh` locally from the release branch and verify all providers/models work +- [ ] Launch goose, click reset providers, choose databricks and a model + +## Starting Conversations + +Test various ways to start a conversation: + +- [ ] Open home and start a new conversation with "Hello" + - [ ] Agent responds + - [ ] Token count is updated after agent finishes + - [ ] Go to history and see there is a new entry +- [ ] Go back to the main screen, start a new conversation from the hub and see that it opens a new conversation +- [ ] Open history and click the Hello conversation - verify it loads +- [ ] Add a new message to this conversation and see that it is added +- [ ] Change the working directory of an existing conversation + - [ ] Ask "what is your working directory?" + - [ ] Response should match the new directory +- [ ] Open a new window, click chat in left side for new chat +- [ ] Click "create a tamagotchi game" in popular chat topics to test developer extension + +## Recipes + +### Create Recipe from Session + +- [ ] Start a simple chat conversation like "hi" +- [ ] Click "create a recipe from this session" in the bottom chat bar + - [ ] Recipe title, description and instructions should be filled in with details from the chat + - [ ] Add a few activities and params (params unused indicator should update if added to instructions/prompts or activities) + - [ ] Can launch create and run recipe - launches in a new window showing as a recipe agent chat with parameters filled in and interact with it + - [ ] Recipe should be saved in recipe library + +### Use Existing Recipe + +- [ ] Pick trip planner from recipe hub (go/gooserecipes) + - [ ] See the warning whether to trust this recipe (only on fresh install) + - [ ] See the form pop up + - [ ] Fill in the form with "Africa" and "14 days" + - [ ] Check results are reasonable + - [ ] Ask how many days the trip is for - should say 14 + +### Recipe Management + +- [ ] Go to recipe manager and enter a new recipe to generate a joke + - [ ] See that it works if you run it + - [ ] Edit the recipe by bottom bar and click "View/Edit Recipe" + - [ ] Make it generate a limerick instead + - [ ] Check that the updated recipe works + - [ ] Delete the recipe from the recipe manager + - [ ] Verify recipe is actually deleted + +### Recipe from File + +- [ ] Create a file `~/.config/goose/recipes/test-recipe.yaml` with the following content: + +```yaml +recipe: + title: test recipe again + description: testing recipe again + instructions: The value of test_param is {{test_param}} + prompt: What is the value of test_param? + parameters: + - key: test_param + input_type: string + requirement: required + description: Enter value for test_param +``` + +- [ ] See that it shows up in the list of installed recipes +- [ ] Launch the recipe, see that it asks for test_param +- [ ] Enter a number, see that it pre-fills the prompt and tells you the value after you hit submit +- [ ] Go to hub and enter "what is the value of test_param" +- [ ] See a new chat that says it has no idea (recipe is no longer active) + +## Extensions + +### Manual Extension Addition + +- [ ] Can manually add an extension using random quotes from project + - [ ] Add new custom stdio extension with the following command and save: + - [ ] `node /ABSOLUTE/PATH/TO/goose/ui/desktop/tests/e2e/basic-mcp.ts` (use your actual project path) + - [ ] Should add and can chat to ask for a random quote + +### Playwright Extension + +- [ ] Install the playwright extension from the extensions hub + - [ ] Tell it to open a browser and search on Google for cats + - [ ] Verify that the browser opens and navigates + +### Extension with Environment Variables + +- [ ] Install an extension from deeplink that needs env variables: + - [ ] Use: `goose://extension?cmd=npx&arg=-y&arg=%40upstash%2Fcontext7-mcp&id=context7&name=Context7&description=Use%20up-to-date%20code%20and%20docs&env=TEST_ACCESS_TOKEN` + - [ ] Extension page should load with env variables modal showing + - [ ] Allow form input and saving extension + +## Settings + +- [ ] Settings page loads and all tabs load +- [ ] Can change dark mode setting + +## Follow-up Issues + +Link any GitHub issues filed during testing: + +--- + +**Tested by:** _____ +**Date:** _____ +**Notes:** _____