# Voice Isolation & Voice Printing

> Learn where audio cleanup (AEC/AGC/noise suppression) happens on a call, and what FirstByte Voice does and does not do for speaker identity.

Clean input audio makes every downstream stage better. Turn detection fires on
real speech instead of HVAC hum. ASR word error rate drops. The agent stops
talking over a caller who never actually spoke. This page tells you **where**
that cleanup happens on a FirstByte Voice call and how to control it. It
also tells you what is *not* built, so you do not design against a feature
that does not exist yet.

The short version: audio cleanup lives at the **edge where audio is
captured**, not in the server media plane. Browser and app callers get full
echo cancellation, automatic gain, and noise suppression for free from the
capture device. Telephony callers rely on carrier-side and handset-side
processing. The server-side media plane is deliberately a codec-and-timing
layer, not a signal-processing (APM) stage.

## Two capture edges, two cleanup stories

  
    When you capture a mic with the SDK's `captureMicrophone()`, the browser's
    own audio pipeline runs a full **AEC + AGC + noise-suppression** chain
    ahead of the Opus encoder. The SDK drives that pipeline through a loopback
    capture graph purely to run the browser's encoder. It then diverts the
    *encoded* Opus frames onto the media-over-QUIC track. You inherit the
    device's processing, and you do not pay for a WebRTC media transport.

    The defaults enable all three. Override them per capture when you have a
    reason. For example, a studio caller on a headset can want raw audio:

    ```ts
    import { captureMicrophone } from "clutchcall_connect/moqt";

    // Defaults: AEC + AGC + noise suppression all ON.
    const cap = await captureMicrophone(publication);

    // Opt out — e.g. an external hardware processor already cleaned the signal.
    const raw = await captureMicrophone(publication, {
      audioConstraints: {
        echoCancellation: false,
        autoGainControl: false,
        noiseSuppression: false,
      },
    });

    // Or bring your own already-processed track.
    const custom = await captureMicrophone(publication, { track: myTrack });
    ```

    > **NOTE:**
> `captureMicrophone()` refuses to run unless the browser exposes the
>       standard encoded-frame transform. Thus cleanup and the encoded path stay
>       consistent for the whole session. See
>       [/modalities/voice/transport-web/browser-audio-capture](/modalities/voice/transport-web/browser-audio-capture)
>       and
>       [/modalities/voice/transport-web/rtcrtpscripttransform](/modalities/voice/transport-web/rtcrtpscripttransform).
  

  
    A call that arrives over a carrier trunk is already **G.711 (µ-law/A-law)
    at 8 kHz** when it reaches the SIP gateway. The server media plane decodes
    it, resamples as needed, and hands PCM16 to the runtime. It does **not**
    apply echo cancellation, gain control, or denoise. Any cleanup on a PSTN
    leg happened upstream: in the caller's handset, in the carrier network, or
    in customer-premises equipment (a PBX/SBC that you migrate from).

    There is no server-side echo canceller on this leg. Barge-in is therefore
    tuned conservatively, so the agent's own text-to-speech cannot leak back
    in and cancel itself. See the interplay note below.
  

## Why the media plane has no APM

The server media plane is a **shard-per-core, zero-copy** path built for
throughput and low, predictable latency. It decodes and encodes codecs,
resamples between clock rates, paces RTP, and bridges legs. It is
intentionally *not* a DSP stage. An inline per-frame audio-processing module
(AEC/AGC/denoise) would add per-hop latency and CPU to a plane that pushes
more than a million packets per second per NIC. It would also duplicate work
that the capture edge already does for free on the browser/app path.

The one place this shows up in behavior is barge-in on telephony:

> **WARNING:**
> On a leg with **no echo cancellation** (raw SIP/PSTN), the agent's own
>   spoken response can bleed into the inbound audio. To stop a self-triggered
>   false interruption, the barge-in gate is **not** lowered while the agent
>   speaks on telephony. The browser AEC path can lower the gate, because the
>   device already removed the echo. If you route a non-AEC transport, keep
>   that gate conservative. Details are in
>   [/modalities/voice/concepts/turn-detection](/modalities/voice/concepts/turn-detection).

## "Who is speaking?" — channel separation vs. voice printing

Two different questions hide under speaker identity. FirstByte Voice
answers only the first one today.

### Channel separation (built in, structural)

You almost never need acoustic diarization to tell the **caller** from the
**agent**. The two legs are already separate streams end to end. They arrive
as distinct RTP sessions / distinct media-over-QUIC tracks. Call recordings
are written **stereo: caller on the left channel, agent on the right**.
Per-speaker transcription, per-side talk-time, and turn attribution come from
the transport, not from a voice model. If your goal is "transcribe each party
separately" or "measure agent vs. caller talk ratio," you already have it.
See
[/modalities/voice/observability/call-traces](/modalities/voice/observability/call-traces)
and
[/modalities/voice/transport-telephony/codecs](/modalities/voice/transport-telephony/codecs).

### Voice printing / speaker ID (not built)

> **NOTE:**
> **Aspirational, not shipped.** FirstByte Voice has **no** speaker-ID,
>   voiceprint/biometric, or acoustic-diarization feature in the media path
>   today. There is no enrollment step, no per-speaker embedding, and no
>   "verify the caller by their voice" API. Voiceprint appears only as a
>   *possible future* authentication factor on the contact-center roadmap. It
>   is not a shipped capability.

If you need speaker verification or diarization **now**, do it at the runtime
layer. Point the call audio at a third-party ASR/biometrics provider that
offers it. Or run diarization on the separated channels of the finished
recording. The runtime's bring-your-own-provider model is the supported seam.
See [/modalities/voice/runtime/byo-asr-llm-tts](/modalities/voice/runtime/byo-asr-llm-tts).

## What you can rely on today

| Capability | Status | Where it runs |
| --- | --- | --- |
| AEC / AGC / noise suppression on browser & app capture | Available | Capture device (browser audio pipeline) |
| Codec decode/encode + resampling on the media plane | Available | Server media plane |
| Stereo caller/agent channel separation in recordings | Available | Media plane recording tap |
| Server-side AEC / AGC / denoise (APM) on telephony legs | Not built | — (rely on carrier/handset) |
| Speaker ID / voiceprint / acoustic diarization | Not built (roadmap) | — (use a BYO provider) |

  - **[Turn detection & barge-in](/modalities/voice/concepts/turn-detection)** — Learn how the no-echo-cancellation media plane shapes the barge-in gate.
  - **[Browser audio capture](/modalities/voice/transport-web/browser-audio-capture)** — See where AEC/AGC/NS actually run for browser and app callers.
  - **[Codecs](/modalities/voice/transport-telephony/codecs)** — Learn about G.711 on the wire, PCM16 in the runtime, and the resampling
    between them.
  - **[Troubleshooting: isolation](/modalities/voice/troubleshooting/voice-isolation)** — Diagnose noisy input, echo, and self-triggered barge-in.