Introducing Eleven Realtime: a simpler way to build natural voice agents

THE NEWS

What happened

ElevenLabs launches 'Eleven Realtime,' a new class of Speech-to-Speech models designed to simplify building natural voice agents. The release introduces two variants: v1 Mini for cost-effective conversational use cases and v1 Max for complex tasks requiring reasoning and tool use. Built on a cascaded architecture, these models eliminate the latency trade-offs of previous fused models while maintaining high accuracy and speed benchmarks against competitors like OpenAI.

CONTEXT

Why it matters

ElevenLabs has introduced Eleven Realtime, a breakthrough in voice agent technology that simplifies development by removing the need to stack individual speech recognition and synthesis components. The launch features two new models: v1 Mini, optimized for natural conversations at low cost, and v1 Max, designed for complex workflows requiring reasoning and tool use. Built on a cascaded architecture, these models offer end-to-end tuning that previously required extensive developer effort. A key advantage is the ability to inspect text outputs at every step, facilitating better guardrails and data privacy controls. Performance benchmarks are impressive: v1 Max answered 98.8% of reasoning questions correctly and started speaking in just 0.57 seconds on average. The system is fully compatible with OpenAI's Realtime API specifications, allowing existing integrations to migrate simply by changing the endpoint.

AT A GLANCE

Key facts

• Eleven Realtime removes the need to stack individual components (ASR, TTS, LLM) by providing complete, tuned voice agent models.
• Two new models are launched: v1 Mini for natural conversations and low cost; v1 Max for complex tasks like customer support and lead qualification.
• The models utilize a cascaded architecture that produces text at every step, enabling guardrails and PII redaction within ElevenAgents.
• v1 Max achieved 98.8% accuracy on Big Bench Audio reasoning questions and started speaking in 0.57 seconds on average.
• Eleven Realtime is compatible with OpenAI's Realtime API specifications, allowing developers to switch endpoints without rewriting code.
• In the Speech Agent Arena, v1 Max scored 998 (human preference) compared to v1 Mini's 988, and completed 87.1% of tool-use tasks versus 60.3% for v1 Mini.