# High Turn Latency

> Diagnose and fix slow agent replies: provider round-trips, region distance, cascade vs speech-to-speech, transcode, and cold models.

Turn latency is the gap your caller feels: the milliseconds between the **end
of the caller's speech** and the **start of the agent's reply**. When it goes
past ~700 ms, the call stops feeling conversational. The caller talks over the
agent, repeats the question, or hangs up. The good news: turn latency is
almost always the sum of a small number of hops, and each hop has a specific
fix.

Work from the top down. First, measure *where* the time goes. Then attack the
biggest hop. Do not guess. The dominant cost is rarely the one you assume.

> **NOTE:**
> This page is about the *turn* budget: the round-trip from silence to the first
> audio back. If the audio is delayed but steady (a fixed offset, or choppy or
> robotic playback), that is a media-plane problem. See
> [One-way / choppy audio](/modalities/voice/troubleshooting/one-way-audio) and
> [MOS, jitter & loss](/modalities/voice/observability/mos-jitter-loss).

## Measure before you tune

Turn latency is a chain, and the system attributes every hop for you. Open the
per-turn [latency breakdown](/modalities/voice/observability/latency-breakdown)
for a recent call. Read the hop-by-hop split before you change anything. A
typical cascaded turn has these hops:

| Hop | What it is | Who owns it |
|-----|-----------|-------------|
| End-of-turn detection | The trailing silence the runtime waits out before it decides the caller is done | Your `turn_detection` config |
| Transport in | Caller audio → relay → runtime | Network + transport |
| ASR finalize | Streaming partials → final transcript | ASR provider |
| LLM time-to-first-token | Prompt sent → first token back | LLM provider |
| TTS time-to-first-audio | Text → first synthesized frame | TTS provider |
| Transport out | First agent frame → caller | Network + transport |

A speech-to-speech (realtime) agent collapses ASR + LLM + TTS into one
provider round-trip, so its breakdown is much shorter. Find your longest hop
in the breakdown. Then go to the matching row below.

## Symptom → cause → fix

1. **Read the breakdown and find the dominant hop**
Open the [latency breakdown](/modalities/voice/observability/latency-breakdown)
for a slow turn. The largest hop is your target. The rest is noise until you
fix it.
2. **Match it to a cause below**
Use the table to map the dominant hop to a root cause and a concrete fix.
3. **Change one thing, re-measure**
Apply one fix. Place a fresh test call. Read the breakdown again. Turn latency
fixes compound, so check each fix before you add the next.

| Symptom (dominant hop) | Likely cause | Fix |
|------------------------|-------------|-----|
| **LLM + ASR + TTS each add a round-trip** | A cascaded pipeline serializes three separate provider hops | Move to a [speech-to-speech provider](#cascade-vs-speech-to-speech) — one round-trip instead of three |
| **One provider hop dominates (ASR *or* LLM *or* TTS)** | That provider's own [round-trip / region](#provider-round-trip) is slow, or the provider is non-streaming | Pick a streaming client, a nearer provider region, or a faster model |
| **The first turn of every call is slow, later turns are fast** | [Cold model / cold session](#cold-models) — the provider connection or self-hosted pod warms up on the first turn | Rely on session prewarm. Keep self-hosted pods warm |
| **Every hop is a bit slow, and the tail is spiky (p99 ≫ p50)** | [Region / relay distance](#region-and-relay-distance), or a TCP/WebSocket fallback leg | Colocate the media plane near the provider. Check that you are on QUIC, not the fallback |
| **Agent replies feel late but the provider hops are fast** | [End-of-turn detection](#end-of-turn-detection) waits out too much silence | Lower `silence_duration_ms` |
| **A small, constant overhead on phone legs only** | [Codec transcode / resampling](#codec-transcode) between G.711, PCM16, and Opus | Usually leave it — it is sub-frame. It only matters as codec passthrough for bridges |

## Cascade vs speech-to-speech

This is the biggest lever. A **cascaded** agent (ASR → LLM → TTS) pays three
serialized provider round-trips per turn. The transcript must finalize before
the LLM starts. The LLM must emit text before TTS can synthesize. A
**speech-to-speech** (realtime) agent streams audio straight into one model
that emits audio back. This collapses all three hops into a single
bidirectional stream. The difference is often a ~1.2 s turn versus a
sub-500 ms turn.

- **[Switch to speech-to-speech](/modalities/voice/runtime/byo-speech-to-speech)** — When to use a realtime model, and how to wire one up. Options include OpenAI
Realtime, Gemini Live, xAI Grok Voice, ElevenLabs Conversational AI, and a
self-hosted realtime model.
- **[Keep a cascade, make it fast](/modalities/voice/runtime/byo-asr-llm-tts)** — If you need the flexibility of separate ASR/LLM/TTS, the fixes below keep a
cascade competitive.

> **TIP:**
> Use a cascade when you need per-stage control: a specific ASR for an accent or
> language, a domain-tuned LLM, or a particular TTS voice. If you do not need
> that control, speech-to-speech is the faster default.

## Provider round-trip

If one hop dominates and it is an individual provider, the latency comes from
that provider's own service, not from us. There are three fixes, in order of
impact:

1. **Use streaming clients, not batch**
Streaming ASR emits partials and finalizes fast. Streaming TTS returns
time-to-first-audio in a few frames instead of after the whole utterance. A
non-streaming HTTP LLM is the worst offender. It returns nothing until it
generates the full response, so the caller waits for the entire completion.
Prefer a streaming or realtime model for any latency-sensitive agent.
2. **Move the provider region closer**
Every provider hop crosses the network twice. If your agent runs in one region
and the provider's endpoint is in another region, that geography is part of
every turn. Point the provider at its region nearest to your media plane.
3. **Pick a faster model**
A smaller or faster-tier model at each stage directly cuts time-to-first-token
and time-to-first-audio. Latency and quality trade off here. Tune per agent.

> **NOTE:**
> A non-streaming LLM also breaks clean barge-in. The runtime can silence
> playback when the caller interrupts, but the provider still completes the full
> response and bills for it. See
> [Turn detection & barge-in](/modalities/voice/concepts/turn-detection) for
> why.

## Region and relay distance

Media rides QUIC / media-over-QUIC to the relay at `relay.firstbyte.fast`,
then continues to your agent runtime and providers. Two distances matter, and
they add up:

- **Caller → relay.** Callers connect to the nearest edge. Browser and app
  legs auto-reconnect with capped backoff if a path degrades. This hop is
  usually small.
- **Runtime ↔ provider.** This is the larger, more variable cost. If your
  agent runtime and your ASR/LLM/TTS providers sit in different regions, a
  cascade pays that crossing three times on every turn.

> **WARNING:**
> **Check that you are on QUIC.** QUIC / media-over-QUIC is the first-class
> transport. WebSocket and WebRTC are *fallbacks* for Safari, UDP-blocked
> networks, and TCP-only proxies. A fallback leg can roughly double turn latency
> and blow out the tail. In a WAN prototype at 50 ms link RTT with ~1% loss,
> turns ran **p50 ~91 ms over QUIC vs ~193 ms over TCP+TLS**, with a far cleaner
> tail. These are prototype-rig numbers, not a production SLA. If your tail
> latency is spiky, check that the leg did not fall back to TCP.

For colocated agents, keep the runtime and the providers it calls in the same
region. The runtime ↔ provider hop then stays local.

## Cold models

The first turn of a call can be slow while everything downstream warms up. The
runtime already fights this for you:

- **Session prewarm.** For realtime agents, the runtime opens and warms the
  provider session *during* the inbound INVITE / SDP exchange, before the
  caller completes the connection. The first turn does not pay connection
  setup.
- **Continuous prefill (self-hosted).** A warm self-hosted realtime model with
  continuous prefill can return time-to-first-audio in roughly **0–15 ms**
  after the session is live.

The failure mode is a **cold self-hosted pod**. If you run your own inference
and the GPU pod scaled to zero, the first call after idle pays the full model
cold-start. The result is a slow first turn, then fast turns after it. Keep a
warm floor of capacity for latency-sensitive traffic.

- **[Self-hosted inference](/modalities/voice/runtime/self-hosted-inference)** — How the on-prem inference control plane routes to your model pods, and how to
keep capacity warm.

## End-of-turn detection

Sometimes the providers are fast and the agent *still* feels late. The cause:
the runtime waits out silence before it decides the caller is done. That wait
is `silence_duration_ms` (default `500`). Every millisecond here adds to the
front of every turn.

- **Lower it (~300 ms)** for faster replies with terse callers.
- **Raise it (~800 ms)** for callers who pause mid-thought, if the agent
  continues to cut them off.

This is a direct latency-vs-interruption trade, not a bug. Tune it against
real calls.

- **[Turn detection & barge-in](/modalities/voice/concepts/turn-detection)** — The full set of end-of-turn and barge-in timings, including `silence_duration_ms`,
`prefix_padding_ms`, and hold-and-confirm barge-in.

## Codec transcode

Phone legs arrive as G.711 (µ-law / A-law). The agent bus runs on narrowband
PCM16. Browser and app legs use Opus. The runtime transcodes and resamples
between these in-process. In practice this cost is **sub-frame and not your
latency problem**. Time spent on it is almost never worth it.

Codec choice does matter in one place: *bridging*. When you keep the same
codec end to end — for example, when you pass Twilio Media Streams µ-law
straight through to a PCMU trunk — you skip transcode on that path. See
[Codecs](/modalities/voice/transport-telephony/codecs) and
[Migrate from Twilio](/modalities/voice/migration/from-twilio) for the
passthrough fast path.

## Related

- **[Latency breakdown](/modalities/voice/observability/latency-breakdown)** — The hop-by-hop attribution to read first — where every millisecond of a turn
goes.
- **[Choose a runtime](/modalities/voice/runtime/overview)** — Cascade vs speech-to-speech vs bring-your-own, and the latency of each.
- **[Barge-in not working](/modalities/voice/troubleshooting/barge-in)** — The other half of turn timing — when the agent does not yield to the caller.
- **[MOS, jitter & loss](/modalities/voice/observability/mos-jitter-loss)** — When the problem is audio quality rather than turn speed.