# On-Prem / Self-Hosted

> Run the voice engine, media relay, and optional self-hosted inference inside your own network.

Run the whole voice stack inside your network, on your hardware: the media
engine, the relay, and (optionally) your own speech models. No part of a call
has to leave your perimeter. SIP trunks terminate on your host. The engine
bridges media in-process. Audio never transits FirstByte's cloud. This is
the path for regulated verticals (HIPAA / SOC 2), data-residency requirements,
and air-gapped or VPC-locked estates.

Self-hosting is **shipped and runs in production** at customer sites today. It
is not yet a one-click installer. The first rollout is script-driven, and our
team usually does it with you. This page maps what runs where and the shape of
the install, so you know what you sign up for.

> **NOTE:**
> Before you self-host, read
>   [Deployment Models](/modalities/voice/concepts/deployment-models) to decide
>   between **managed cloud**, **fully self-hosted**, and **hybrid** (self-hosted
>   media plane + managed control plane). Many teams need only the media plane
>   on-prem. The control plane can stay managed.

## What runs where

A self-hosted voice deployment is two hosts (plus an optional GPU host):

| Role | What it does | Where it runs |
|------|--------------|---------------|
| **Voice engine** | Terminates SIP trunks and RTP media, runs the QUIC/HTTP-3 transport on `:443`, embeds the AI agent runtime, and bridges browser audio. One process, one machine. | A dedicated Linux host with its own NIC |
| **Media relay** | Fans MoQT audio tracks out to browser/app subscribers and across hosts. Fronted at `relay.firstbyte.fast`. | The same host (single-site) or separate hosts (multi-site mesh) |
| **Control plane** | The tenant admin API, the agent/trunk config store, the call-events pipeline, and recording storage. Serves `portal.firstbyte.fast`, the Contact Center console at `agent.firstbyte.fast`, and the Voice AI console at `voiceai.firstbyte.fast`. | A second box (DB + object store + event bus) |
| **Inference** (optional) | Your own speech-to-speech / ASR-LLM-TTS model server, reached over QUIC. | GPU host(s) — see below |

The engine is a modular host process. The SIP gateway (B2BUA + registrar), the
RTP media plane, the QUIC/MoQT engine, the relay, and the agent runtime are all
modules loaded into one process. You deploy and operate the engine as a single
unit.

## Before you begin

  1. **A dedicated Linux host for the engine**
The engine takes **full ownership of its NIC** with a kernel-bypass fast
    path (zero-copy RX/TX for SIP, RTP, and QUIC on one interface). For that
    reason, the NIC cannot be shared with other services. Give the engine its
    own bare-metal box, or a VM with a dedicated, passthrough-capable
    interface.

    > **WARNING:**
> Because the process owns the NIC, a config change or a new build is a
>       **reboot cutover**, not a hot reload. While the engine is down, the
>       interface is quiesced. While the engine runs, you do not see a normal
>       listening `UDP :443` socket in `ss`. Plan a maintenance window for
>       changes. This is the single biggest operational difference from a
>       typical service.
  2. **A control-plane host**
A second Linux box runs the admin API, the config database, the
    call-events store, and recording object storage. There is **no Docker**.
    Everything runs as `systemd` units against a pinned Node runtime under
    `/opt`. This keeps the install auditable, and it avoids a container
    runtime in regulated environments.
  3. **(Optional) a GPU host for self-hosted models**
If you want inference on-prem too, provision one or more GPU hosts to
    serve your speech model. The engine reaches them through the inference
    control plane over QUIC. See
    [Self-Hosted Inference](/modalities/voice/runtime/self-hosted-inference).
  4. **DNS you control**
You point per-tenant SIP and WebRTC hostnames at the engine. You point the
    console/relay hostnames at the correct box. Public QUIC/WebTransport needs
    valid TLS certs for your brand domains (the transport does SNI across
    them).

## Install and cut over

  1. **Provision the hosts**
Stand up the engine host (dedicated NIC) and the control-plane host. If
    you self-host models, also stand up the GPU host(s). Lay down the
    `systemd` units and the service config on each host.
  2. **Deploy the engine**
The deploy script builds the engine host, its transport core, and the
    modules. The script ships them to the box and swaps them in atomically.
    Then **reboot** the engine host to bring the NIC-owned data plane up on
    the new build (see the reboot warning above).
  3. **Bring up the control plane**
Start the admin API, the database, the event bus, and the object store on
    the control-plane host as `systemd` units. The event bus is always
    TLS + SASL authenticated. There is no plaintext mode. Recordings land in
    the local object store and show in the console.

    > **NOTE:**
> Tenant credentials (trunk digest secrets, provider API keys) are
>       **sealed at rest**. The engine and the control plane share one
>       key-encryption key, so the engine can unseal the credentials at call
>       time. No plaintext secrets sit on disk. This is what makes the
>       deployment HIPAA / SOC 2 defensible.
  4. **Point DNS and provision a tenant**
Create a workspace in the console. Onboarding provisions the per-tenant
    SIP and WebRTC hostnames — `<workspace-id>.sip.firstbyte.fast` and
    `<workspace-id>.webrtc.firstbyte.fast` — and the SIP `SRV` records that
    route carrier `INVITE`s to your engine. Point `portal.firstbyte.fast`,
    `agent.firstbyte.fast`, and `relay.firstbyte.fast` at the correct hosts.
  5. **(Optional) attach self-hosted inference**
Register your model server with the inference control plane. Set the
    agent's provider to your endpoint. The engine then routes turns to your
    GPUs over QUIC instead of a cloud realtime provider. Full steps are in
    [Self-Hosted Inference](/modalities/voice/runtime/self-hosted-inference).
  6. **Place a test call**
Register a softphone against your tenant SIP domain (or point a carrier
    trunk at it). Dial in. Watch the call land and the agent answer. If
    recording is on, watch the `.wav` appear in the console. See
    [Inbound Calls](/modalities/voice/transport-telephony/inbound) for the
    trunk and dialplan wiring.

## Honest status

> **NOTE:**
> **What's shipped:** the engine, the SIP/RTP media plane, the QUIC/MoQT
>   transport, the relay, the agent runtime, the call-events + recording
>   pipeline, and credential sealing all run in production on-prem today.
> 
>   **What's still maturing:** the **self-hosted inference** leg. The QUIC-based
>   control plane that discovers and routes to your GPU pods is shipped. However,
>   the fully on-prem speech-to-speech model path is not yet live-verified
>   end-to-end in a customer deployment. If you need on-prem GPU inference from
>   day one, plan it as a joint bring-up, not a drop-in. The managed-cloud model
>   providers (cloud realtime, ASR/LLM/TTS) work against a self-hosted engine
>   today, if your network allows egress to them.

> **NOTE:**
> There is intentionally **no separate sandbox environment**. A self-hosted
>   deployment is a single production environment. Test in a dedicated workspace
>   or a second install, not a sandbox toggle.

## Related

  - **[Deployment Models](/modalities/voice/concepts/deployment-models)** — Managed, self-hosted, and hybrid — how to split the media and control planes.
  - **[Self-Hosted Inference](/modalities/voice/runtime/self-hosted-inference)** — Serve your own speech models on your GPUs and route to them over QUIC.
  - **[Managed Cloud](/modalities/voice/quickstart/managed-cloud)** — Prefer zero infra? Ship the same agent on the managed cloud in minutes.
  - **[Self-Hosted Troubleshooting](/modalities/voice/troubleshooting/self-hosted)** — NIC ownership, reboot cutovers, DNS, and sealed-credential gotchas.