# Browser Audio Capture

> Capture the microphone in the browser, encode it to Opus, and play received audio back. You never touch raw PCM.

A browser tab can act as a voice endpoint: it turns the microphone into
Opus frames you can publish over QUIC, and it turns received Opus frames back
into sound. The SDK ships two helpers that do exactly this on top of a live
transport session: `captureMicrophone(publication, opts)` for the send side,
and `OpusPlayer` for playback. You never touch WebCodecs or raw PCM by hand.

The capture side reuses the browser's own audio pipeline. A loopback
connection runs the built-in echo cancellation, automatic gain control, and
noise suppression, then the platform's Opus encoder. You tap the **encoded**
frames off the encoder and send the bytes. Nothing decodes the audio in the
page. The browser does the DSP and the codec; you only move the payload. See
[WebRTC diversion](/modalities/voice/transport-web/webrtc-diversion) for why
this design keeps you off a media SFU, and
[RTCRtpScriptTransform](/modalities/voice/transport-web/rtcrtpscripttransform)
for how the tap works.

> **NOTE:**
> The encoded-frame tap runs only in a Worker, through the standard
> `RTCRtpScriptTransform`. If the browser lacks it, `captureMicrophone` **refuses
> to capture**. It does not fall back to an insecure path, and it never acquires
> a mic it cannot use. Check support first with `encodedTransformSupported()`.
> Read [Browser compatibility](/modalities/voice/transport-web/browser-compatibility)
> for the support matrix.

## Capture the microphone

1. **Connect a media session**
Open a transport session to your relay and set `requireEncodedTransform: true`.
That option enforces the encoded-frame rule for the whole session up front. On
an unsupported browser, you fail at connect time, not after you acquire the
mic.

```ts
import { MoqtClient, encodedTransformSupported } from "@clutchcall/agents";

if (!encodedTransformSupported()) {
  // Show a fallback UI; this browser can't send audio through the tap.
  throw new Error("Encoded audio capture unsupported in this browser");
}

const client = await MoqtClient.connect(
  "quic://relay.firstbyte.fast",
  tenantToken,
  (state) => console.log("transport:", state),
  { requireEncodedTransform: true },
);
```

2. **Publish an audio track**
Create an outbound audio track. `publishAudio(ns, name, opts)` returns an
`AudioPublication` whose `write(timestampUs, frame)` accepts one encoded Opus
frame at a time. Namespaces are room-scoped by call. Key them off your
`call_sid` (for example, namespace `voice/<sid>`, track name `uplink`).

```ts
const uplink = client.publishAudio(`voice/${callSid}`, "uplink", {
  capability: "agent.mic.opus",
});
```

The Opus track runs 48 kHz mono, 20 ms frames on the wire. You do not set the
rate; the browser's encoder does. Those are the defaults an
`AudioPublication` carries.

3. **Start capture**
Hand the publication to `captureMicrophone`. It prompts for the mic (with
AEC/AGC/NS enabled), runs the encoder, installs the transform Worker, and calls
`uplink.write(...)` for every encoded frame. It returns a handle. Call
`stop()` on the handle to tear the capture graph down.

```ts
import { captureMicrophone } from "@clutchcall/agents";

const capture = await captureMicrophone(uplink);

// later, when the user mutes or hangs up:
capture.stop();
```

> **WARNING:**
> `capture.stop()` tears down the capture graph (Worker, loopback connection,
> and the mic track it acquired) but does **not** close the MoQT publication or
> the transport session. Close those yourself when the call ends.

### Capture options

`captureMicrophone(publication, opts)` takes an optional `MicCaptureOptions`:

- **`audioConstraints`** (`MediaTrackConstraints`) — Constraints passed to `getUserMedia({ audio })`. Defaults enable `echoCancellation`, `autoGainControl`, and `noiseSuppression`. Override to pin a device or relax the DSP chain.

- **`track`** (`MediaStreamTrack`) — Supply your own audio track instead of a `getUserMedia` call — for example, a track you already processed. When you pass this, `stop()` leaves the track running (you own its lifecycle).

> **NOTE:**
> The browser's echo cancellation, gain control, and noise suppression are the
> **only** cleanup on the send side. The server-side media path does not run AEC
> or AGC of its own. On PSTN legs, where there is no browser AEC, the runtime
> compensates with voice-activity gating instead (see
> [Turn detection](/modalities/voice/concepts/turn-detection)).

## Play received audio

`OpusPlayer` is the receive-side mirror. WebCodecs decodes the Opus frames you
receive. An `AudioWorklet` ring buffer plays them out and pads with silence
on underrun. Feed the player the frames delivered by `subscribeAudio`.

```ts
import { OpusPlayer } from "@clutchcall/agents";

const ctx = new AudioContext();
const player = new OpusPlayer(ctx);
await player.start(); // load the worklet + configure the decoder once

client.subscribeAudio(`voice/${callSid}`, "downlink", (tsUs, opus) => {
  player.push(tsUs, opus);
});

// on hangup:
player.close();
```

`OpusPlayer` accepts an options object: `sampleRate` (decoder output rate,
default 48000) and `channels` (default 1). Call `start()` once before the first
`push`, and `close()` to release the decoder and disconnect the worklet.

> **WARNING:**
> Browsers gate `AudioContext` behind a user gesture. Create the context (or call
> `ctx.resume()`) from a click or tap, or playback stays silent until the user
> interacts with the page.

## A minimal two-way softphone

Put both halves together for a complete browser voice leg: publish the mic,
subscribe to the far side, and play it.

```ts
import {
  MoqtClient,
  captureMicrophone,
  OpusPlayer,
  encodedTransformSupported,
} from "@clutchcall/agents";

async function joinCall(callSid: string, token: string) {
  if (!encodedTransformSupported()) throw new Error("unsupported browser");

  const client = await MoqtClient.connect(
    "quic://relay.firstbyte.fast",
    token,
    undefined,
    { requireEncodedTransform: true },
  );

  // send: mic → Opus → uplink track
  const uplink = client.publishAudio(`voice/${callSid}`, "uplink");
  const capture = await captureMicrophone(uplink);

  // receive: downlink track → Opus decode → speakers
  const ctx = new AudioContext();
  const player = new OpusPlayer(ctx);
  await player.start();
  client.subscribeAudio(`voice/${callSid}`, "downlink", (ts, f) =>
    player.push(ts, f),
  );

  return () => {
    capture.stop();
    player.close();
    void ctx.close();
  };
}
```

> **NOTE:**
> These helpers are **browser-only**: `getUserMedia`, `RTCRtpScriptTransform`,
> and WebCodecs have no server-side equivalent. To send or receive audio from
> Node, Python, Go, or Rust, use the native-core audio bridge instead. See the
> [voice SDKs](/modalities/voice/sdk/javascript). Native mobile capture is
> [plan-only today](/modalities/voice/transport-web/mobile-app).

> **NOTE:**
> In the browser, this transport is **WebTransport-only**. There is no automatic
> QUIC→WebSocket fallback rung yet (that rung ships in the native SDK cores). If
> a network blocks QUIC, the media session fails; it does not downgrade. Plan
> your fallback UX around
> [WebTransport availability](/modalities/voice/transport-web/webtransport).

## Next steps

- **[WebRTC diversion](/modalities/voice/transport-web/webrtc-diversion)** — Why capture reuses the browser's WebRTC pipeline as a codec tap, not a
transport.
- **[RTCRtpScriptTransform](/modalities/voice/transport-web/rtcrtpscripttransform)** — How the Worker taps encoded Opus frames and where E2EE fits in.
- **[Browser compatibility](/modalities/voice/transport-web/browser-compatibility)** — Which engines run the transform, WebCodecs, and the capture worklet.
- **[Turn detection](/modalities/voice/concepts/turn-detection)** — How on-device VAD and barge-in decide when the caller is speaking.