# Latency Breakdown

> Where the milliseconds go on a voice turn — capture, encode, transport, runtime, egress — with real transport numbers and how to measure your own.

A caller feels exactly one latency number: how long the line stays silent
after they stop talking, before the agent starts to talk back. This page
explains that number. It breaks the number into hops that you can attack one
at a time. It also shows you how to measure your own number instead of trust
in ours.

We call it **turn latency**: the wall-clock gap from **end-of-user-speech** to
**first audio-out**. It is the sum of five hops: capture, encode, transport,
runtime, and egress. There is also one part that people forget to count: the
endpointing window that your turn detector waits before it *decides* the user
is done.

## The five hops

A turn walks the same path in both directions. The user's audio goes
capture → encode → transport → runtime. The reply comes back runtime →
transport → egress. The table shows where the time goes and what dominates
each hop.

| Hop | What happens | What dominates it | Measured how |
| --- | --- | --- | --- |
| **Capture** | Mic → frames. The browser runs AEC/AGC/NS + a jitter buffer. A phone leg is G.711 off the wire | One frame of buffering (**20 ms**) + the browser's own jitter buffer | Client-side frame timestamps |
| **Encode** | Opus (browser/app legs) or G.711 (phone legs). The runtime bus is 8 kHz PCM16 | The codec frame boundary. Opus is 20 ms/frame, `ts_delta = 960` on a 48 kHz clock | SDK capture-to-publish delta |
| **Transport** | Encoded frames ride MoQT over WebTransport/QUIC on a single `:443`, or RTP for phone legs | One network **RTT/2** each way; the congestion state | Per-connection transport telemetry (RTT, cwnd) |
| **Runtime** | ASR → LLM → TTS (cascaded) **or** one speech-to-speech model | Almost always the LLM's time-to-first-token, or the S2S model's prefill | Runtime per-node tracing |
| **Egress** | Reply PCM → 20 ms pacer → Opus/G.711 back to the caller | The paced 20 ms tick; one frame of look-ahead | Egress pacer timestamps |

> **NOTE:**
> The runtime hop dominates the total by an order of magnitude. The transport
> and codec hops are single- to low-double-digit milliseconds. A cascaded
> ASR→LLM→TTS turn is hundreds of milliseconds. If you chase latency, start at
> the runtime. Tune transport only after that. This page gives you both, in
> that order.

## Capture and encode

On a browser or app leg, the audio pipeline is the browser's own. The platform
reuses it and does not replace it. A loopback `RTCPeerConnection` runs echo
cancellation, gain control, noise suppression, the Opus encoder, and the
jitter buffer. An `RTCRtpScriptTransform` in a Worker taps the **encoded**
Opus frames and hands them to the transport. See
[WebRTC diversion](/modalities/voice/transport-web/webrtc-diversion) for the
capture path in detail. The floor here is one Opus frame (**20 ms**) plus the
depth that the browser's jitter buffer settles to on your network.

On a phone leg there is no browser pipeline. G.711 arrives in 20 ms RTP
packets. The media plane decodes it to 8 kHz PCM16. **Noise suppression** runs
on ingress before the audio reaches the runtime. There is deliberately **no
acoustic echo cancellation or AGC** in the media path. PSTN legs use
[VAD-gating](/modalities/voice/concepts/turn-detection) instead. This is a
*correctness* choice, not a latency choice.

> **WARNING:**
> Do not confuse the **20 ms tick** (160 samples at 8 kHz) with turn latency.
> The tick is the frame cadence that the whole pipeline paces on. It bounds
> jitter, not the turn. A single tick is ~20 ms. A turn is hundreds of
> milliseconds, because the model must think.

## Transport: QUIC earns its place here

This is the hop where our transport choice shows up as a number. We used a
controlled prototype rig over a WAN link (**50 ms RTT, ~1% loss**). We
measured the same turn — end-of-user-speech to start-of-agent-speech — over
QUIC and over a TCP+TLS baseline:

| Turn latency (prototype rig, 50 ms RTT, ~1% loss) | QUIC | TCP + TLS |
| --- | --- | --- |
| p50 | **91.2 ms** | 193.0 ms |
| p99 | **91.9 ms** | 193.8 ms |

QUIC roughly **halves** the turn. More important, QUIC keeps the tail flat.
The p99 sits right on the p50, because head-of-line blocking is gone. Under
concurrency (c = 20) the gap widens where it hurts most. Time-to-first-token
p50 is **93.9 ms vs 143.6 ms** (−35%). The p99 is **149.2 ms vs 1197.9 ms**,
an ~8× better tail. Throughput is ~60% higher.

> **WARNING:**
> These are **prototype** numbers from a purpose-built rig. They are not a
> measurement of the production stack. Treat them as directional evidence for
> the transport choice, not as a latency SLA. The productionized transport lives
> on the same `:443` QUIC plane. Measure your own turn (below) before you quote
> a figure. Cost and scale figures live in
> [Benchmarks](/modalities/voice/benchmarks).

### The browser diversion tax

The platform reuses the browser's encoder and ships the tapped frames over
QUIC instead of native SRTP/UDP. This is not free, but it is close to free. In
the localhost diversion prototype, the JS-layer round trip measured **1.20
ms** over QUIC and **0.84 ms** for raw WebRTC SRTP/UDP. The tap and the QUIC
hop cost **+0.36 ms**. That is noise next to a single Opus frame. In exchange
you get a single `:443` plane, no SFU, and encoded-frame E2EE readiness.

## Runtime: where the turn is actually spent

The runtime hop dominates the turn. There are two shapes:

  
    Three serial stages. The clock is:
    - **ASR finalize.** How long after end-of-speech the transcriber emits a
      final result. Streaming ASR overlaps most of this time while the user
      still talks.
    - **LLM time-to-first-token.** Usually the single largest contributor.
      You tune it with model choice, prompt size, and
      [self-hosted inference](/modalities/voice/runtime/self-hosted-inference).
    - **TTS time-to-first-audio.** How fast the first synthesized frame comes
      back. Streaming TTS starts to speak before the sentence is done.

    You do not pay all three end-to-end, because the stages pipeline. But
    first audio cannot leave until ASR finalized *and* the LLM produced
    enough tokens for TTS to start.
  
  
    One model consumes audio and emits audio. It collapses ASR/LLM/TTS into a
    single pass. This is structurally the lowest-latency shape. With a
    self-hosted speech-to-speech model kept warm with continuous prefill,
    time-to-first-audio can reach **~0–15 ms** after the turn is committed.
    See [BYO speech-to-speech](/modalities/voice/runtime/byo-speech-to-speech).

    The catch is turn ownership. Some realtime providers own turn detection
    server-side (`type: "server_vad"`). A self-hosted model with no server
    VAD needs a client-commit gate to tell it the user is done. That gate
    puts the endpointing window (next section) back on your critical path.
  

## The hidden hop: endpointing

Before any of the five hops runs, the turn detector must *decide* that the
user stopped. That decision has a cost. It is real latency that the caller
feels:

- **`silence_threshold_ms`.** How long a pause must last before it counts as
  end-of-turn. If you lower it, the agent responds faster but cuts people off
  mid-thought. If you raise it, the agent feels sluggish. This is the single
  biggest knob that you own.
- **Hold-and-confirm barge-in.** A speech onset arms a *pending* barge. The
  barge fires only if speech continues past a short window (~300 ms). Thus a
  backchannel ("mm-hm") does not cancel the agent. This trades a little
  barge-in latency for an agent that does not talk over the caller.

Tune these in [Turn detection](/modalities/voice/concepts/turn-detection).
Know that a "slow agent" complaint is often an endpointing setting, not a
transport or model problem.

## Measure your own turn

Do not trust our numbers. Instrument yours. Turn latency is a
client-observable quantity. You have both timestamps. Thus you can compute it
without any server-side access.

  1. **Stamp end-of-speech**
Record the wall-clock time when your turn detector commits the turn (the
    `Commit` decision, or the provider's server-VAD end-of-turn signal). This
    is `t0`.
  2. **Stamp first audio-out**
Record the arrival time of the first downlink audio frame on
    `voice/<sid>/downlink`. The SDK hands you each frame. This is `t1`.
    **Turn latency = `t1 − t0`.** Log it per turn. Take p50/p95/p99 over a
    call. A single number lies.
  3. **Split transport out**
Read the per-connection **RTT** and congestion window from the QUIC
    transport telemetry that the platform exports to your analytics. If turn
    latency drifts up but RTT is flat, the regression is in the runtime, not
    the network.
  4. **Split the runtime stages**
The agent runtime emits per-node timings as tracing spans: ASR finalize,
    LLM time-to-first-token, and TTS time-to-first-audio. Subtract them from
    your `t1 − t0`. The remainder is capture + transport + egress.

> **NOTE:**
> There is **no packaged "turn-latency" dashboard panel** today. You assemble
> the view from the three signals above: client-side `t1 − t0`, exported
> transport RTT, and runtime node spans. The quality dashboards that *do* ship —
> [MOS, jitter & loss](/modalities/voice/observability/mos-jitter-loss) —
> measure media quality, not turn latency. They complement each other. They are
> not substitutes.

## When the number is too high

| Symptom | Likely cause | Fix |
| --- | --- | --- |
| The agent feels sluggish, RTT is flat | The endpointing window is too long | Lower `silence_threshold_ms`. Check the result with `t1 − t0` |
| Turn latency is high, runtime spans are small | Transport RTT/loss | Check QUIC RTT/cwnd. Check that UDP/`:443` is reachable and that there is no TCP-proxy fallback |
| The first token is slow, everything else is fine | LLM time-to-first-token | Use a smaller or warm model. Trim the prompt. See [self-hosted inference](/modalities/voice/runtime/self-hosted-inference) |
| A long tail (p99 ≫ p50), p50 is fine | Head-of-line blocking on a TCP path | Check that the QUIC/WebTransport rung is in use, not a WS/TCP fallback |
| The agent talks over the caller | The barge-in confirm is too aggressive | Raise the hold-and-confirm window. See [barge-in debugging](/modalities/voice/observability/barge-in-debugging) |

The full runbook is in
[High latency](/modalities/voice/troubleshooting/high-latency).

## Related

  - **[High latency runbook](/modalities/voice/troubleshooting/high-latency)** — Step-by-step diagnosis when turn latency is too high.
  - **[MOS, jitter & loss](/modalities/voice/observability/mos-jitter-loss)** — Media-quality metrics. They are separate from turn latency. The page
    tells you how to read them.
  - **[Call traces](/modalities/voice/observability/call-traces)** — Follow a single call end to end across every hop.
  - **[Turn detection](/modalities/voice/concepts/turn-detection)** — Endpointing and barge-in: the knobs that shape the hidden hop.
  - **[Runtime overview](/modalities/voice/runtime/overview)** — Cascaded vs speech-to-speech: the hop that dominates the turn.
  - **[Benchmarks](/modalities/voice/benchmarks)** — How the stack compares on latency, cost, and scale.