Voice AI latency: what to measure before you judge a call
A low-latency demo is not proof of a responsive phone experience. Measure the caller’s pause to first audible reply, interruptions, and the full turn under real network conditions.
Vistrow Voice team
Product, speech, and customer experience
The pause callers notice
For a caller, latency is the silence between finishing a thought and hearing the agent begin its response. That wait includes audio transport, speech recognition, language-model processing, text-to-speech generation, and playback buffering.
Provider benchmarks often measure only one part of that chain. For a fair comparison, measure from the end of the caller’s turn to the first audible agent audio in the same channel you plan to deploy.
A language switch can change recognition and synthesis time, so use the multilingual voice agent evaluation checklist for mixed-language calls. For visitors speaking from a browser, the website voice widget guide covers permission and connection steps that a phone-only test misses.
Run a repeatable test
Use the same script, language, network, and question set for each configuration. Record several calls rather than relying on one best-case result. Test both short answers and longer responses, then review the audio alongside timestamps and logs.
- Track median and slower-turn latency, not only the fastest call.
- Separate greeting time from response time after the caller speaks.
- Test interruptions, silence detection, barge-in, and end-of-turn behavior.
- Repeat over browser audio and the actual phone route; their network paths differ.
- Log the selected speech, model, and fallback route so a test is attributable.
Tune the whole pipeline
Shorter, well-structured answers can reduce perceived waiting and keep a conversation focused. Streaming recognition and speech generation can help when supported, but buffering, slow tool calls, or synchronous work on the audio event loop can erase those gains.
Optimize from observed traces, one component at a time. A useful target is a conversation that feels responsive and remains accurate—not the smallest number in a provider’s marketing table.
Make a timing sheet before changing providers
For each turn, note when the caller finishes speaking, when the system commits the turn, when the first model text arrives, when the first synthesized audio arrives, and when that audio becomes audible. These timestamps are not interchangeable. A fast text response can still sit behind a slow audio buffer.
If most of the delay appears before the model starts, changing the language model may do very little. Inspect end-of-turn detection and speech recognition first. If the model responds quickly but speech starts late, look at text chunking, synthesis startup, transport, and playback. Use server traces and a call recording together; either one alone misses part of the experience.
Use a comparison that cannot hide the slow turns
Run at least a few repeated conversations per configuration and keep the failures in the results. Report a median and a slower percentile, along with sample size and the channel. Do not mix browser and phone observations into one number or compare a cached greeting with a fresh answer.
Include a short question, a longer explanation, an interrupted answer, and a tool-backed request. A system can feel fast until it checks availability. Also listen for premature replies: cutting a caller off may shorten the measured pause while making the conversation worse. Choose the fastest configuration that still lets people finish and gets the task right.
- Record exact provider, model, voice, region, and fallback.
- Keep the same prompt and response-length instructions.
- Measure the actual phone route as well as the browser.
- Separate tool time from model and synthesis time.
- Track misunderstood answers alongside latency.
Technical references
Documentation behind the technical details. The examples and checklists above are our implementation guidance.