# FirstByte Voice Architecture > Follow one call end to end through the SIP gateway, RTP media plane, agent runtime, MoQT transport, and relay mesh. Follow one call from the carrier's edge to your AI agent and back. That path is the whole architecture. FirstByte Voice is one path made of five roles. Each role owns one job and hands the call to the next role: - **SIP gateway / B2BUA** — The gateway terminates carrier trunks and browser-originated calls. It authenticates and routes each call. It joins the two legs of every call. - **RTP media plane** — The media plane receives and sends the audio packets. It transcodes codecs, handles jitter and DTMF, and taps recordings. All of this runs in the engine, per session. - **Agent runtime** — The runtime drives the conversation. It streams caller audio to your ASR/LLM/TTS or speech-to-speech provider. It paces the reply back into the call. - **MoQT transport** — The transport carries audio to and from browsers and apps as media-over-QUIC objects. The objects tunnel through WebTransport on the same port as your control API. A **relay mesh** sits under the transport. It fans each audio track out to every subscriber across engine cores and hosts. You do not wire these roles together yourself. You point a trunk or a browser client at the stack, and the path assembles itself. ## The end-to-end path Here is an inbound PSTN call that reaches an AI agent. Read it top to bottom. Each arrow is a handoff between the roles above. ``` Carrier SIP trunk │ INVITE (matched by source IP → tenant + trunk) ▼ ┌─────────────────────────┐ │ SIP gateway / B2BUA │ digest auth · ACL · spam filter · dialplan └───────────┬─────────────┘ │ route to agent, open media on the caller's core ▼ ┌─────────────────────────┐ │ RTP media plane │ G.711 (PCMU/PCMA) 8 kHz → decode to PCM16 └───────────┬─────────────┘ │ 8 kHz PCM16 frames (20 ms / 160 samples) ▼ ┌─────────────────────────┐ │ Agent runtime │ ASR → LLM → TTS, or speech-to-speech └───────────┬─────────────┘ │ agent audio, 8 kHz PCM16 ▼ ┌─────────────────────────┐ │ RTP media plane egress │ pacer → G.711 encode → RTP back to carrier └─────────────────────────┘ ``` 1. **The call arrives** The carrier sends an `INVITE` to the tenant's SIP edge (`.sip.firstbyte.fast`). The gateway matches the trunk by source IP and resolves the tenant. It runs digest auth, ACL, and spam filtering before it accepts the call. 2. **The gateway routes and opens media** The gateway acts as a back-to-back user agent (B2BUA). It picks the destination from the dialplan and skill-based routing. It then opens an RTP media session pinned to the CPU core that owns the call. 3. **The media plane decodes** Carrier audio is G.711 µ-law or A-law at 8 kHz. The media plane decodes it to the runtime's internal format (8 kHz PCM16, little-endian) in 20 ms frames. It also handles jitter, DTMF, and the recording tap. 4. **The runtime holds the conversation** Decoded audio streams into the agent runtime. The runtime runs your cascaded ASR → LLM → TTS graph or a single speech-to-speech provider. It decides turn boundaries and produces reply audio. 5. **The reply flows back** Reply audio goes through the media plane's egress pacer. The media plane re-encodes the audio to the carrier's codec and sends it back over RTP. The gateway holds the SIP dialog open until either side hangs up. Everything inside the runtime runs at **8 kHz PCM16** end to end. A call crosses at most one transcode on the way in and one on the way out. See [Sessions, calls, tracks & streams](/modalities/voice/concepts/sessions-calls-tracks-streams) for how these map to the object model. ## Direct media vs. proxied media You choose per trunk whether the media plane and the SIP gateway sit on the same host. Set `media_mode` on the trunk: RTP terminates where the agent runs. After call setup, the gateway is **signalling-only**. It manages the SIP dialog while audio flows directly from the carrier to the runtime's media plane. This mode has fewer hops, lower latency, and no media relay to scale. Use this mode for well-behaved carriers. ``` Carrier ──RTP audio──▶ RTP media plane ─▶ Agent runtime ▲ │ └── SIP signalling ──▶ SIP gateway (dialog only, no media) ``` The gateway relays RTP through its media path before the audio reaches the runtime. This mode adds one hop. But the carrier only sees one media address. Carriers behind hostile NAT or a locked-down SBC continue to work. ``` Carrier ──RTP──▶ SIP gateway (media relay) ──▶ RTP media plane ─▶ Agent runtime ``` > **NOTE:** > Direct media is the fast path and the default. Use `proxy` only when a > carrier's NAT or SBC policy refuses to send audio to an address other than > the one it signalled to. ## Browser and app clients A browser or mobile client does not speak SIP. It uses the media-over-QUIC transport. Audio never touches WebRTC's transport. The SDK borrows only the browser's Opus encoder and capture graph (see [WebRTC diversion](/modalities/voice/transport-web/webrtc-diversion)): ``` Browser mic ─▶ Opus encode ─▶ MoQT object ─▶ WebTransport :443 │ ▼ media-over-QUIC bridge Opus decode · 48k→8k resample │ ▼ bridge into the call / agent Browser player ◀─ Opus ◀─ MoQT track ◀─ 8k→48k ◀─ caller / agent audio ``` Uplink and downlink use **distinct MoQT tracks**: a `mic` track up and a `spk` track down. Thus a client never subscribes to its own audio, and no feedback loop occurs. The relay mesh does the fan-out. See [Transport to web & apps](/modalities/voice/transport-web/overview) for the full detail. ## Two planes on the QUIC edge The engine terminates a single QUIC listener that multiplexes two logical planes: | Plane | Protocol | Carries | Reached at | |-------|----------|---------|------------| | **Control** | QUIC + HTTP/3 | Session setup, agent config, call control, telemetry | `portal.firstbyte.fast` / `agent.firstbyte.fast` | | **Media** | MoQT over WebTransport | Live audio tracks (publish / subscribe) | The same `:443`, tunneled through WebTransport | For browsers, both planes converge on one `:443`. MoQT rides inside WebTransport next to the HTTP/3 control traffic. You open one port and one certificate. Native SDKs can also dial a dedicated media endpoint for the MoQT plane. > **NOTE:** > The shared `:443` WebTransport port is the current direction for the media > plane. A standalone media-over-QUIC port still exists for native SDKs that > dial the transport directly. That port is the legacy option, not the > default. WebSocket (control and data) and WebRTC (browser media) stay the > required fallbacks when QUIC or WebTransport is unavailable. ## The relay mesh The engine runs one reactor per CPU core (shard-per-core). The engine pins each call to a core. When a track must reach subscribers on other cores or on other hosts, the **relay mesh** does the fan-out: - **Cross-core** — the mesh broadcasts a published track to sibling cores on the same host through a lock-free ring. A subscriber pinned to a different core still receives every audio object. - **Cross-host** — edges peer with each other and push tracks between hosts. A browser connected to one edge can subscribe to a call that terminates on another edge. You can reach the mesh at `relay.firstbyte.fast`. The mesh lets one call share one audio track set between a PSTN caller on one core, an AI agent, and a browser-based human supervisor on a different host. None of them must know where the others run. ## Related - **[Components](/modalities/voice/concepts/components)** — See each piece of the stack and the job it owns. - **[Sessions, calls, tracks & streams](/modalities/voice/concepts/sessions-calls-tracks-streams)** — Learn the object model behind the data path. - **[Transport to telephony](/modalities/voice/transport-telephony/overview)** — Learn about SIP trunking, numbers, ACD, and the call lifecycle. - **[Agent runtime](/modalities/voice/runtime/overview)** — Bring your own ASR/LLM/TTS or speech-to-speech provider. - **[Deployment models](/modalities/voice/concepts/deployment-models)** — Compare managed cloud and on-prem self-hosting. - **[Transport to web & apps](/modalities/voice/transport-web/overview)** — Learn how browser and mobile clients reach the stack over QUIC.