✳flâneur — a map of the web's best reading

How we solved latency at Vapi

vapi.ai · 1,684 words · saved by 1 readers

In real-time voice applications, the most important metric is latency to response, measured as the duration between a user’s end of statement and the agent’s start of statement. This cycle is called turn-taking. Conversational flow breaks when latency exceeds 1200ms. That’s the rough time it takes for the user to have a tangential thought. This gives us a strict 1200ms latency budget for every turn in a conversation. We treat this budget like a scarce resource. If we can save milliseconds in the LLM reasoning step, we can spend them on a higher-fidelity TTS model for a more human-like voice. Here’s how that 1200ms budget is typically spent in a speech-to-speech pipeline: The ASR (speech-to-text) and TTS (text-to-speech) models are fairly optimized by their underlying providers. The bottleneck is almost always the LLM.. specifically, the time to first meaningful sentence. LLM providers advertise impressive speeds, but their benchmarks rarely hold up in production. We tracked OpenAI’s G

How we solved latency at Vapi - Vapi AI Blog Skip to main content Vapi raises $50M Series B to power the next generation of enterprise voice AI Vapi raises $50M Series B Read More → How we solved latency at Vapi Abhishek Sharma • Jul 14, 2025 3 min read Share Latency is the enemy of conversational flow. In real-time voice applications, the most important metric is latency to response , measured as the duration between a user’s end of statement and the agent’s start of statement. This cycle is called turn-taking . Conversational flow breaks when latency exceeds 1200ms. That’s the rough time it

Explore this link on the map →

related reading