AWS Shows How to Stream Kokoro TTS on SageMaker AI for Voice Agents
Disclosure: this post contains affiliate links. If you buy through them we may earn a commission at no extra cost to you. Learn more.
TL;DR: AWS published a technical guide showing how to stream speech synthesis from AI voice agents using the Kokoro-82M text-to-speech model on Amazon SageMaker AI. The approach eliminates multi-second delays by streaming each sentence to the TTS model as the agent produces it, rather than waiting for the complete response. The 82-million parameter Kokoro model runs on a single mid-tier GPU instance to keep costs and latency low.
What's New
Amazon Web Services published a technical how-to guide on October 22, 2026, demonstrating how to reduce latency in AI voice agents by streaming text-to-speech synthesis sentence-by-sentence. The solution uses the Kokoro-82M text-to-speech model hosted on Amazon SageMaker AI.
According to AWS, traditional voice agent integrations create noticeable delays because they wait for an agent's complete response before beginning speech synthesis. A one-sentence response might generate in 600 milliseconds, but a full multi-paragraph response can take five seconds, during which the customer hears nothing. This creates a frustrating user experience where customers wait in silence even though the agent has already started generating an answer.
The demonstrated solution streams each sentence to the hosted Kokoro-82M model as soon as the agent produces it, allowing audio playback to begin before the agent finishes its full response. The Kokoro-82M model contains only 82 million parameters, making it small enough to run on a single mid-tier GPU instance while maintaining low latency and inference costs.
The implementation runs Kokoro-82M in a custom container on a SageMaker AI real-time endpoint using an ml.g5.xlarge GPU instance. The container exposes two routes: a POST endpoint for streaming responses to complete inputs, and a WebSocket endpoint that maintains one connection open across multiple messages in both directions. The guide covers both Server-Sent Events and unified binary transport methods for delivering audio back to browsers.
Pricing and Availability
The solution is available now on Amazon SageMaker AI. AWS does not provide specific pricing in the guide, though costs will depend on the ml.g5.xlarge GPU instance usage for the SageMaker AI real-time endpoint. The guide notes that the small 82-million parameter size of Kokoro-82M helps keep both latency and inference costs low for voice agents handling real traffic.
How to Try It
AWS provides deployment code through a GitHub repository referenced in the blog post. Prerequisites include an AWS account with permissions to create SageMaker AI endpoints and Amazon Bedrock AgentCore runtime, AWS Cloud Development Kit (CDK) v2, Python 3.12 or later, and familiarity with WebSockets.
Deployment involves two CLI tools wrapping AWS CDK stacks. The first command deploys the Kokoro-82M endpoint, including foundation and model-cache stacks that create IAM roles and sync model weights into Amazon S3. The second command provisions the Amazon Bedrock AgentCore runtime hosting the Strands agent.
The guide includes detailed technical walkthroughs for handling WebSocket frame types, maintaining persistent sessions between messages, and implementing an IncrementalSentenceStreamer class that normalizes text arriving as either complete strings or live token streams. The solution requires a SageMaker AI real-time endpoint deployed from a container declaring specific Docker labels for bidirectional streaming support.
AWS notes an important caveat: the sample container uses an older PyTorch version with a known security advisory (PYSEC-2026-2286, fixed in PyTorch 2.10.0). Users should review and upgrade before any production use.
Who It's For
This solution targets developers building AI voice agents who need to minimize perceived latency in customer interactions. It's particularly relevant for organizations already using AWS infrastructure who want to improve the responsiveness of conversational AI applications.
The technical implementation requires familiarity with AWS services, WebSocket protocols, and ASGI frameworks like Starlette, making it most appropriate for experienced cloud developers rather than beginners.
Alternatives
While AWS demonstrates Kokoro-82M specifically, organizations can explore other text-to-speech solutions for voice agents. The guide focuses on the streaming architecture rather than being exclusive to one TTS model, suggesting the approach could work with other models hosted on SageMaker AI. Developers building conversational AI might also consider workflow automation platforms like Make for orchestrating multi-step agent interactions.
Tools mentioned
FAQ
What latency improvement does sentence-by-sentence streaming provide?
According to AWS, traditional integrations can create delays of five seconds or more while waiting for a complete multi-paragraph response before starting synthesis. Streaming each sentence as produced allows audio playback to begin immediately after the first sentence completes, eliminating most of this perceived delay.
What AWS services are required for this solution?
The solution requires Amazon SageMaker AI for hosting the Kokoro-82M model endpoint, Amazon Bedrock AgentCore for the agent runtime, Amazon S3 for storing model weights, Amazon ECR for the container repository, and IAM for permissions management.
Can this approach work with text-to-speech models other than Kokoro-82M?
While AWS demonstrates the solution specifically with Kokoro-82M, the guide focuses on the streaming architecture. The approach of streaming sentences to a TTS model as an agent produces them could potentially work with other models hosted on SageMaker AI, though AWS does not explicitly confirm compatibility with other models.
What are the hardware requirements for running Kokoro-82M?
AWS runs Kokoro-82M on an ml.g5.xlarge GPU instance in SageMaker AI. The 82-million parameter model is specifically sized to run on a single mid-tier GPU instance to keep both latency and inference costs manageable for production voice agent traffic.