Realtime voice, speech, and transcription now supported on AI Gateway
By Steven Van ·
The beta, available through AI SDK 7, adds the same spend controls and bring-your-own-key support Vercel offers for text, image, and video models.
Vercel's AI Gateway now supports voice and audio models, in beta via AI SDK 7. Realtime voice agents, text-to-speech, and speech-to-text get the same observability, spend controls, and bring-your-own-key support as the Gateway's text, image, and video models, with no markup or platform fees.
- Realtime voice agents: a single model listens to a user, works out a response, and speaks it back in a live, low-latency conversation, and can call tools mid-conversation to look something up or take an action. The useRealtime hook handles microphone capture and playback.
- Text to speech: generates spoken audio from text, with a selectable voice and output format such as MP3, for voiceovers or spoken responses.
- Speech to text: transcribes a file buffer, base64 string, or URL into text, for voice notes or other transcriptions.
Realtime sessions are capped at 25 minutes, close after 5 minutes of inactivity or if nothing arrives within 30 seconds of connecting, and reject messages larger than 256 KB. The docs use openai/gpt-realtime-2 and spacexai/grok-voice-think-fast-1.0 as examples; the latter handles speech-to-speech only, with no transcription or translation.