mirror of
https://github.com/block/goose.git
synced 2026-07-17 12:56:20 +02:00
docs: voice dictation updates (#7396)
This commit is contained in:
dianed-square
@@ -237,23 +237,30 @@ Speak to goose directly instead of typing your prompts.
|
||||
1. Click the <PanelLeft className="inline" size={16} /> button in the top-left to open the sidebar
|
||||
2. Click `Settings` in the sidebar
|
||||
3. Click the `Chat` tab
|
||||
4. Under `Voice Dictation`, toggle `Enable Voice Dictation` on
|
||||
5. Choose between `OpenAI Whisper` or `ElevenLabs` as your dictation provider
|
||||
6. Enter your API key for the provider you chose
|
||||
4. For `Voice Dictation Provider`, select your provider from the dropdown:
|
||||
- Local — On-device transcription using a local whisper model, no API key needed. On first use, you'll be prompted to download the model.
|
||||
- [Elevenlabs](https://elevenlabs.io/)
|
||||
- [Groq](https://groq.com/)
|
||||
- [OpenAI](https://platform.openai.com/api-keys)
|
||||
5. If prompted, enter your API key for the provider you chose
|
||||
|
||||
To use voice dictation:
|
||||
1. Return to your chat (click your chat session under `Chat` in the sidebar)
|
||||
2. Click the microphone on the right of the chat box and begin speaking
|
||||
1. Click your chat session under `Chat` in the sidebar
|
||||
2. Click the microphone button on the right of the chat box and begin speaking
|
||||
3. To send your message, do one of the following:
|
||||
- Say "submit" to send the message and continue recording your next one. To stop recording, click the microphone button.
|
||||
- Click the microphone button to stop recording, then click `Send` or press `Enter`. This option allows you to edit the message before sending.
|
||||
|
||||
The first time you use voice dictation, goose will request access to your microphone. While recording, you'll see a live waveform of your audio in the input field, a timer, and the current size of your recording. Click the microphone button again to finish recording.
|
||||
The first time you use voice dictation, goose will request access to your microphone. While recording, you'll see `Listening` and `Transcribing` status indicators. goose transcribes your speech during natural pauses and adds the text to the chat box.
|
||||
|
||||
**If you don't see the microphone**, check the [models you have configured](/docs/getting-started/providers.md). ElevenLabs can be used as a dictation provider alongside any LLM, but OpenAI Whisper requires that you have an OpenAI model configured in goose, even if using another LLM provider for chat.
|
||||
**If you don't see the microphone**, check the [models you have configured](/docs/getting-started/providers.md). For example, using OpenAI as the voice dictation provider requires that you have an OpenAI model configured in goose, even if using another LLM provider for chat.
|
||||
|
||||
#### Important Notes
|
||||
* You can record up to 10 minutes or 25MB of audio
|
||||
* The audio is processed by your chosen provider (OpenAI or ElevenLabs)
|
||||
* Voice input is appended to any existing text in the text input field, so you can combine typing and speaking your prompts
|
||||
* You can record up to 50MB of audio
|
||||
* ElevenLabs, Groq, and OpenAI send audio recordings to their servers for processing. The local provider processes audio entirely on your device—no data leaves your computer.
|
||||
* Voice input is appended to any existing text in the text input field
|
||||
* Recordings are not stored locally after transcription
|
||||
* To disable voice dictation, select `Disabled` from the provider dropdown
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="cli" label="goose CLI">
|
||||
|
||||
@@ -152,6 +152,23 @@ curl x POST http://localhost:3000/api/users \
|
||||
Would you like me to add any additional features or make other improvements to the setup?
|
||||
```
|
||||
|
||||
## Environment Variables in Shell Commands
|
||||
|
||||
Shell commands executed by the `shell` tool inherit the environment of the running goose process. This typically includes:
|
||||
- System variables like `PATH`, `HOME`, and `USER`
|
||||
- Environment variables present in the process that launched goose (for example, your terminal's environment when you start goose from a shell)
|
||||
- Session-specific variables injected by goose, such as `AGENT_SESSION_ID` for [session-isolated workflows](/docs/guides/environment-variables#using-session-ids-in-workflows)
|
||||
|
||||
This enables workflows that depend on environment configuration, such as authenticated CLI operations and build processes.
|
||||
|
||||
:::info
|
||||
goose Desktop or launcher-based starts may use a different environment and may not load your shell startup files.
|
||||
:::
|
||||
|
||||
:::warning Sensitive Information
|
||||
Environment variables may contain sensitive values like API keys and tokens (e.g., `GITHUB_TOKEN`, `AWS_ACCESS_KEY_ID`).
|
||||
:::
|
||||
|
||||
## Configuring Access Controls
|
||||
|
||||
By default, goose can run system commands with your user privileges and edit any accessible file **without your approval**. This is because goose runs in Autonomous permission mode by default and has access to the Developer extension's shell and file editing tools. While this configuration allows goose to work quickly and independently, there's potential for unexpected outcomes. Understanding the available access control features can help you configure goose to match your comfort level and specific needs.
|
||||
@@ -172,23 +189,6 @@ The Developer extension provides these tools:
|
||||
| `screen_capture` | Take screenshots | Debugging UI issues, documenting state | ✅ Low<br />Visual information only |
|
||||
| `image_processor` | Process and resize images | Optimizing assets, format conversion | ✅ Low<br />Image manipulation only |
|
||||
|
||||
### Environment Variables in Shell Commands
|
||||
|
||||
Shell commands executed by the `shell` tool inherit the environment of the running goose process. This typically includes:
|
||||
- System variables like `PATH`, `HOME`, and `USER`
|
||||
- Environment variables present in the process that launched goose (for example, your terminal's environment when you start goose from a shell)
|
||||
- Session-specific variables injected by goose, such as `AGENT_SESSION_ID` for [session-isolated workflows](/docs/guides/environment-variables#using-session-ids-in-workflows)
|
||||
|
||||
This enables workflows that depend on environment configuration, such as authenticated CLI operations and build processes.
|
||||
|
||||
:::info
|
||||
goose Desktop or launcher-based starts may use a different environment and may not load your shell startup files.
|
||||
:::
|
||||
|
||||
:::warning Sensitive Information
|
||||
Environment variables may contain sensitive values like API keys and tokens (e.g., `GITHUB_TOKEN`, `AWS_ACCESS_KEY_ID`).
|
||||
:::
|
||||
|
||||
### Access Control Features
|
||||
|
||||
You can layer multiple controls to match your risk tolerance and workflow:
|
||||
@@ -240,3 +240,24 @@ As you become more comfortable with goose's behavior, you can adjust these setti
|
||||
:::info
|
||||
Also see the [Security Guide](/docs/guides/security/) for information about using goose safely.
|
||||
:::
|
||||
|
||||
## Additional Resources
|
||||
|
||||
import ContentCardCarousel from '@site/src/components/ContentCardCarousel';
|
||||
|
||||
<ContentCardCarousel
|
||||
items={[
|
||||
{
|
||||
type: 'topic',
|
||||
title: 'Enhanced Code Editing with AI Models',
|
||||
description: 'Use AI models to intelligently apply code changes',
|
||||
linkUrl: '/goose/docs/guides/enhanced-code-editing'
|
||||
},
|
||||
{
|
||||
type: 'topic',
|
||||
title: 'Codebase Analysis',
|
||||
description: 'Understand codebases with semantic analysis and call graphs',
|
||||
linkUrl: '/goose/docs/guides/codebase-analysis'
|
||||
}
|
||||
]}
|
||||
/>
|
||||
|
||||
Reference in New Issue
Block a user