# LiveKit Agent over FirstByte Transport

> Run an unmodified LiveKit agent over FirstByte. Install the transport plugin into your agent process (recommended), front-proxy the media through the voice gateway, or use the browser compat shim.

You already have a LiveKit agent that works. You want callers to reach it over
the PSTN, or you want its media to use FirstByte's QUIC transport instead of
a LiveKit SFU. You do not want to rewrite the agent. There are three ways to do
that. The recommended way is a **package that you install into your agent
process**.

  - **Transport plugin** — **Recommended.** `pip install` / `npm i` a plugin into your existing
    `livekit-agents` worker. Your agent code, plugins, and provider keys do not
    change. Only the transport changes. There is no LiveKit server at all.
  - **Front-proxy the media** — The voice gateway terminates the caller. It bridges audio into your
    existing LiveKit **room** through a sidecar. Use this path when you must
    keep your LiveKit server in operation.
  - **Browser compat shim** — **Preview.** Change one import. A `livekit-client` **browser** app then
    publishes and subscribes over MoQT/QUIC instead of the SFU.

> **NOTE:**
> These three paths solve different problems. The **plugin** replaces LiveKit's
>   transport for a server-side **agent**. The **front-proxy** keeps your LiveKit
>   server and puts FirstByte in front of it. The **compat shim** is for a
>   **browser** client. The paths combine. You can front-proxy a phone leg and
>   move a browser client onto the transport at the same time.

## Path A — the transport plugin (recommended)

Install the plugin next to your agent. The plugin wraps `AgentSession.start`.
It binds the session's audio input and output to a QUIC/WebTransport connection
to FirstByte instead of a LiveKit room. The LiveKit Agents SDK skips its own
`RoomIO` when a session already has custom audio IO set. Thus your `entrypoint`
runs **verbatim**: the same `AgentSession`, the same STT/LLM/TTS plugins, the
same provider keys, and the same tool calls. There is no room, no SFU, and no
WebRTC.

> **INFO:**
> This is the same seam that LiveKit itself uses for its TCP console mode
>   (`TcpAudioInput` / `TcpAudioOutput`). The plugin points that seam at our QUIC
>   edge.

```
 PSTN / SIP trunk ─▶ ┌──────────────────┐                   ┌────────────────────────────┐
                     │  voice gateway   │  ONE QUIC/WT      │ your livekit-agents process │
 browser (QUIC/MoQT) │  (owns the       │◀── session ──────▶│  + transport plugin         │
 on :443 ──────────▶ │  caller leg)     │  (agent dials out)│  (your STT/LLM/TTS keys)    │
                     └──────────────────┘                   └────────────────────────────┘
```

Your agent is always the **QUIC client**. It dials out to `:443` and opens no
inbound ports. Thus it works behind NAT and needs no public address. One
session carries three lanes:

- presence/registry (liveness)
- audio over QUIC **datagrams**
- per-call control (job assign, flush, barge-in, playback finished) over a
  WebTransport stream

### Install

  
```bash
pip install "livekit-plugins-clutchcall[quic]"
```

Current version **0.0.9**. Requires Python 3.9+ and `livekit-agents>=1.0`. The
`quic` extra installs `aioquic` (the WebTransport client). Without it, the
package still imports, but it cannot connect.
  
  
```bash
npm i @clutchcall/livekit-transport @fails-components/webtransport
```

Current version **0.1.7**. Requires Node 20+. `@livekit/agents` (tested against
1.6.1) and `@livekit/rtc-node` stay as your own dependencies — the plugin
declares them as **optional peers** and never pins your SDK version.
  

### Run your agent over the transport

Replace your worker's entrypoint runner (`cli.run_app(...)` /
`cli.runApp(...)`) with ours. The code it calls, your `entrypoint`, does not
change.

  
```python worker.py
import asyncio, os
from livekit.plugins import clutchcall

from my_agent import entrypoint   # your own file, unedited

asyncio.run(clutchcall.run_agent_worker(
    clutchcall.WebTransportConfig(
        engine_url="https://engine.firstbyte.fast",
        app_key=os.environ["CLUTCHCALL_MEDIA_KEY"],
        app_secret=os.environ["CLUTCHCALL_MEDIA_SECRET"],
        agent_name="salesbot",     # how the engine addresses this worker
        sample_rate=8000,          # telephony bus; use 48000 for wideband legs
    ),
    host="engine.firstbyte.fast",
    entrypoint=entrypoint,
))
```
  
  
```ts worker.ts
import { runAgentWorker, defaultConfig } from "@clutchcall/livekit-transport";
import { entrypoint } from "./my-agent.js";   // your own file, unedited

const host = "engine.firstbyte.fast";

await runAgentWorker(
  defaultConfig({
    engineUrl:  `https://${host}`,
    appKey:     process.env.CLUTCHCALL_MEDIA_KEY!,
    appSecret:  process.env.CLUTCHCALL_MEDIA_SECRET!,
    agentName:  "salesbot",   // how the engine addresses this worker
    sampleRate: 8000,         // telephony bus; use 48000 for wideband legs
  }),
  { host, port: 443, entrypoint },
);
```
  

`run_agent_worker` does these tasks:

1. It authenticates one connection.
2. It registers the worker under `agent_name`.
3. It serves the calls that the engine assigns to it. For each call, it starts
   your `entrypoint` with a `JobContext` bound to that call's media.

If the connection drops, the worker reconnects with exponential backoff. Thus
an engine restart does not leave the worker on a dead socket.

> **WARNING:**
> **Set `sample_rate` to match the call leg.** The `pcm16` codec is a raw
>   passthrough. A wideband TTS voice (24 kHz or 48 kHz) sent onto an 8 kHz
>   telephony leg plays at the wrong speed. The plugin declares the wire rate on
>   the session output, and the SDK resamples for you. But this works only if
>   you set the correct rate.

### Point a FirstByte agent at the worker

The engine side models a remote agent as a **`VENDOR_BRIDGE`** node with
`vendor_provider` set to `clutchcall`. This means external agent
orchestration, and FirstByte stays the transport. `vendor_room` is the
`agent_name` that your worker registered.

```json agent-config.json
{
  "entry_node": "bridge",
  "pipeline": {
    "bridge": {
      "node_type": "VENDOR_BRIDGE",
      "vendor_provider": "clutchcall",
      "vendor_room": "salesbot"
    }
  }
}
```

| Field | Meaning |
| --- | --- |
| `vendor_provider` | `clutchcall`: your agent runs on our transport. The values `livekit` \| `vapi` \| `twilio` front-proxy a vendor instead. |
| `vendor_room` | The `agent_name` that your worker registered |

With this config, the runtime runs **no** turn detection and **no** model in
the engine. It sends the caller's audio to your worker. It streams your agent's
audio back onto the call. Attach the agent to a number or trunk in the same way
as a native agent. See
[SIP Trunking](/modalities/voice/transport-telephony/sip-trunking) and
[Number Provisioning](/modalities/voice/transport-telephony/number-provisioning).

### What your agent sees

Your `entrypoint` receives a `JobContext`. Its `ctx.room` is a **veneer**, not
a LiveKit room. The veneer carries the fields that agents read:

| Field | Value |
| --- | --- |
| `ctx.room.name` | The dispatched agent/call name. This is the identity, thus "get agent by room" continues to work |
| `ctx.job.id` | The platform **call sid** — the id your CDR, transcripts, recordings, and usage rows are all keyed by. Log it to join your own traces to ours |
| `ctx.job.metadata` | Your agent's context metadata as a JSON string (see below). Empty string when none is set |

Caller and called numbers surface where LiveKit's SIP integration puts them —
but **the two plugins put them in different places today**:

  
```python
# Python (livekit-plugins-clutchcall 0.0.9): SIP attributes are on the
# REMOTE PARTICIPANT. `ctx.room` has no `.attributes` on this plugin.
p = next(iter(ctx.room.remote_participants.values()), None)
ani  = p.attributes.get("sip.phoneNumber")        if p else ""   # caller
dnis = p.attributes.get("sip.trunkPhoneNumber")   if p else ""   # called
```

| Surface | Value |
| --- | --- |
| `…remote_participants[*].attributes["sip.phoneNumber"]` | Caller number (ANI) |
| `…remote_participants[*].attributes["sip.trunkPhoneNumber"]` | Called number (DNIS) |
| `ctx.room.attributes` | **Not present** on the Python plugin |
| `sip.trunkID` | **Not mapped** on the Python plugin |
  
  
```ts
// TypeScript (@clutchcall/livekit-transport 0.1.7): SIP attributes are
// on ctx.room, plus flat convenience fields.
const ani   = ctx.room.attributes["sip.phoneNumber"];
const dnis  = ctx.room.attributes["sip.trunkPhoneNumber"];
const trunk = ctx.room.attributes["sip.trunkID"];
// also: ctx.callerNumber, ctx.calledNumber, ctx.trunkId, ctx.callId
```

| Surface | Value |
| --- | --- |
| `ctx.room.attributes["sip.phoneNumber"]` | Caller number (ANI) |
| `ctx.room.attributes["sip.trunkPhoneNumber"]` | Called number (DNIS) |
| `ctx.room.attributes["sip.trunkID"]` | The trunk the call arrived on |
| `ctx.callerNumber` / `ctx.calledNumber` / `ctx.trunkId` | The same three, unwrapped |
  

> **WARNING:**
> This asymmetry is a known gap, not a design: the Python veneer is being
>   brought up to the TypeScript shape. Read the numbers through the surface
>   your plugin actually exposes rather than the one the other tab shows.

An agent that reads the caller's number therefore continues to work without
changes, as long as it reads it the way that plugin surfaces it.

**Context metadata.** Set key-value pairs on the agent (console, or the admin
SDK's `agents.updateConfig` with `{ patch: { metadata: { crm_pipeline:
"inbound-2026" } } }`). The engine forwards them on every call dispatched to
your worker, and the plugin surfaces them where LiveKit puts dispatch metadata:

```python
import json

async def entrypoint(ctx):
    meta = json.loads(ctx.job.metadata) if ctx.job.metadata else {}
    # meta == {"crm_pipeline": "inbound-2026", "order_id": "A-4417"}
```

The agent's pairs are the DEFAULT layer. One call can add to them, or replace a
key, with the `metadata` field of
[`telephony.originate`](/modalities/voice/api/calls) — the per-call value wins.
Use this for context that changes from call to call: an order id, a campaign, a
script revision.

Metadata is not a tag. Tags are the platform's own dimension: they are
inherited down the parent chain, they show in reporting, and they arrive
separately as `call.tags`. Metadata is free-form, applies to this agent only,
and nothing on the platform reads it. Put environment, team and cost centre in
tags. Put your own application context in metadata.

Values are flat strings (numbers and booleans are stringified; nested objects
are dropped), capped at 64 keys / 8 KB per agent. Room **APIs**
beyond these fields are not backed: track publication, participant enumeration,
and room data messages. There is no room behind the veneer.

The plugin forwards finalized transcript turns to the engine automatically over
the control stream. Thus a remote agent's conversation history lands in the
same store as a native agent's history. It shows in
[Observability](/modalities/voice/observability/overview) together with CDR,
MOS, and recording.

### Tool calls and token analytics

Your agent runs its own tool loop and its own STT/LLM/TTS on your provider keys,
so the engine can't see either — but you can forward both over the control
stream and they land on the FirstByte timeline (and fire the
`voice.agent.tool_called` / `voice.agent.usage` webhooks) alongside native
agents. Wire them from your agent's hooks:

```python
from livekit.agents import metrics

# In your entrypoint, after building the session:
@session.on("metrics_collected")
def _on_metrics(ev):
    m = ev.metrics
    conn = clutchcall.current_connection()   # the MediaConnection for this call
    call_id = clutchcall.current_call_id()
    if isinstance(m, metrics.LLMMetrics):
        conn.send_usage(call_id, "llm",
                        {"prompt_tokens": m.prompt_tokens,
                         "completion_tokens": m.completion_tokens},
                        provider="openai", model="gpt-4o")
    elif isinstance(m, metrics.STTMetrics):
        conn.send_usage(call_id, "stt", {"audio_seconds": m.audio_duration})
    elif isinstance(m, metrics.TTSMetrics):
        conn.send_usage(call_id, "tts", {"characters": m.characters_count})

# And from a function tool, so the call is logged:
conn.send_tool_call(call_id, "get_order_status",
                    {"order_id": order_id}, {"status": status})
```

The TypeScript plugin exposes the same methods (`sendUsage`, `sendToolCall`).
**Redact anything sensitive yourself** — args, results and usage fields are
forwarded as-is (the engine can't see inside your agent to redact for you).
Native FirstByte agents get tool logging automatically; usage metering for
native agents is on the roadmap.

### Outbound calling (single + batch)

The same worker serves calls the platform dials out. Trigger dials with the
LiveKit-shaped API in the plugin — it wraps `telephony.originate` on the
administration API, authenticated with an org-scoped `mpk_…` key (mint one
in the portal under **Settings → API Keys** with scope `mcp:telephony:write`; set it as
`CLUTCHCALL_API_KEY` alongside your org id):

```python
from livekit.plugins import clutchcall

api = clutchcall.ClutchCallAPI()   # env: API key + org id

# One call. The engine dials the number on your trunk; when the callee
# answers, your registered worker receives the job_assign like an inbound
# call — with ctx.room attributes carrying the dialed number.
res = await api.sip.create_sip_participant(
    sip_trunk_id="my-trunk",
    sip_call_to="+919986398327",
    agent_id="71156156",
)

# Batch — an engine-paced campaign (calls-per-second + max concurrency
# enforced server-side; watch or stop with list_campaigns / abort_campaign).
await api.sip.create_sip_campaign(
    name="renewals",
    numbers=["+91...", "+91..."],
    sip_trunk_id="my-trunk",
    agent_id="71156156",
    calls_per_second=2,
    max_concurrent_calls=10,
)
```

The TypeScript plugin exposes the same surface (`new ClutchCallAPI()`,
`api.sip.createSipParticipant(...)`, `api.sip.createSipCampaign(...)`). A
runnable example lives in the starter repo as `src/outbound.py`.

Campaign pacing, limits, control (`list_campaigns` / `abort_campaign`), and
what the worker receives on answer are covered in full in
[Outbound Calls from a LiveKit Agent](/modalities/voice/recipes/livekit-outbound).
For the rest of the administration surface use the
[Admin API SDK](/sdks/admin-sdk).

### Reaching the agent from a browser

The same worker also serves **web** callers. `@clutchcall/agent-widget` is an
embeddable widget whose media runs on MoQT/QUIC. The engine republishes the
browser's mic track onto your worker's media plane and returns the reply on the
same downlink, so your agent code sees a web caller exactly as it sees a phone
caller. See
[Embed an Agent Widget on Your Website](/modalities/voice/recipes/web-embed).

### Camera and keyboard input

A browser caller can also send their **camera** and their **keyboard**, and your
worker receives either one — but only if it asks. Declare what you accept and
read it:

  
```python worker.py
asyncio.run(clutchcall.run_agent_worker(
    clutchcall.WebTransportConfig(
        # …as above…
        accepts_video=True,        # ask for the caller's camera
        accepts_text=True,         # …and their typed turns
    ),
    host="engine.firstbyte.fast", entrypoint=entrypoint,
))
```

```python my_agent.py
async def entrypoint(ctx):
    ctx.on_video(lambda jpeg: my_vision_queue.put_nowait(jpeg))
    ctx.on_text(lambda text: my_turn_queue.put_nowait(text))
    # …the rest of your agent is unchanged…
```
  
  
```ts worker.ts
await runAgentWorker(
  defaultConfig({ /* …as above… */ acceptsVideo: true, acceptsText: true }),
  { host, port: 443, entrypoint },
);
```

```ts my-agent.ts
export async function entrypoint(ctx: any) {
  ctx.onVideo((jpeg: Uint8Array) => myVisionQueue.push(jpeg));
  ctx.onText((text: string) => myTurnQueue.push(text));
}
```
  

Camera frames are **whole JPEGs at about 1 fps**, passed through untouched —
nothing in the path decodes, scales or re-encodes them, so what you get is what
the browser captured, ready to base64 into a vision model's input event. The
engine paces them because a realtime provider bills per frame.

Typed turns arrive as plain text, capped at 4 KB. The engine unwraps the wire
envelope, so you get the words, not a schema. The mic and the keyboard are two
inputs to **one** call — your reply goes back as audio either way.

> **NOTE:**
> **Why a callback and not `session.input.video`.** LiveKit's session wants
>   decoded `rtc.VideoFrame`s for a continuous track. These are stills, so feeding
>   that seam would mean shipping a decoder to undo work nobody did. Decode them
>   yourself if you want them on the session.

Three things follow from the capability being *declared*, not assumed:

- **Default off.** A worker that does not ask gets no frames, and the engine
  does not subscribe the track at all — an audio-only agent costs exactly what
  it did before.
- **The caller is told the truth.** The engine announces `video`/`text` for the
  call from what the worker that took it declared, so the widget shows a control
  only when pressing it does something.
PSTN callers have neither a camera nor a keyboard, so both are browser-caller
features.

### Recordings, CDR, and transcripts

Recording is a **call-layer** capture, so a call your worker handles is
recorded exactly like a native agent's — both legs, same storage, same API.
Fetch the audio with the [Recordings API](/modalities/voice/api/recordings),
keyed by the `call_sid` your agent sees as `ctx.job.id`.

### Current limits

  **Python and TypeScript only**

`livekit-plugins-clutchcall` (Python) and `@clutchcall/livekit-transport` (TypeScript) track the
    two LiveKit Agents SDKs. There is no Go or Rust plugin.

  **Audio is pcm16 today**

The wire codec is raw little-endian `pcm16`. Opus fits into the same codec
    seam without changes to the session or IO classes, but it is not connected
    yet. Thus budget bandwidth for uncompressed audio between your worker and
    the edge.

  **Register-and-wait is the shipped launch mode**

The worker holds a connection open, and the engine dispatches calls to it.
    This mode is correct for always-on workers on a VM or in Kubernetes, and it
    is warm on the first call. A per-call HTTP trigger is not yet available.
    That trigger would serve scale-to-zero serverless workers that dial back on
    demand.

  **Audio needs QUIC egress**

The worker needs outbound UDP/443 to the edge. The WebSocket fallback
    covers presence only. Media over TCP reintroduces head-of-line blocking and
    is a degraded last resort. Agents run on server infrastructure, where QUIC
    egress is the norm.

## Path B — front-proxy the media

Use this path when you **must keep your LiveKit server in operation**. For
example, other participants join the room, or another part of your stack
depends on it. In this path, the gateway owns the caller edge: a PSTN SIP
trunk, or a browser leg over QUIC on the single `:443` plane. The gateway
decodes the edge to the runtime's 8 kHz PCM audio bus. It relays that audio
over a WebSocket media bridge to a **bridge sidecar that you run** next to your
LiveKit deployment. That sidecar joins your room as an ordinary participant.

```
 PSTN / SIP trunk ─▶ ┌──────────────────┐  WebSocket   ┌──────────────┐   ┌──────────────┐
                     │  voice gateway   │   media      │ bridge       │   │  your LiveKit│
 browser (QUIC/MoQT) │  (terminates the │──── bridge ──▶ sidecar      │──▶│  room + agent│
 on :443 ──────────▶ │  caller edge)    │              │ (joins room) │   │              │
                     └──────────────────┘              └──────────────┘   └──────────────┘
```

> **NOTE:**
> **What is shipped and what you deploy.** The gateway-side bridge is in-tree
>   and shipped. It consists of a vendor-bridge pipeline node plus the WebSocket
>   media handler. The **bridge sidecar** that joins your LiveKit room is a
>   component that you run next to your LiveKit deployment. It uses your LiveKit
>   URL, API key, and secret.

The agent config is the same `VENDOR_BRIDGE` shape. Here `vendor_provider`
names the vendor instead of FirstByte:

```json agent-config.json
{
  "entry_node": "bridge",
  "pipeline": {
    "bridge": {
      "node_type": "VENDOR_BRIDGE",
      "vendor_provider": "livekit",       // livekit | vapi | twilio
      "vendor_room": "support-room",
      "vendor_creds_ref": "livekit-prod"  // sealed LIVEKIT_URL / key / secret
    }
  }
}
```

The full runnable version is in
[Bridge a LiveKit Agent](/modalities/voice/cookbooks/bridge-livekit). It covers
how to build the sidecar, store credentials, route a number, and place a call.

## Path C — the browser compat shim (preview)

If your LiveKit surface is a **browser app** (`livekit-client`), you can run
the client's audio over our transport directly. The `livekit-compat` shim
re-implements the `livekit-client` API (`Room`, `RoomEvent`, tracks,
participants) on top of MoQT/QUIC. The common voice path is a one-line import
change. That path covers these actions: publish the mic, subscribe to remote
audio, and exchange data. There is no SFU, and the shim never uses WebRTC's
transport.

> **WARNING:**
> **Preview / experimental.** The `livekit-compat` shim is an early-access
>   browser package, not a shipped product. It covers the core voice path:
>   publish/subscribe audio and data messages. **Video, screenshare, and
>   per-participant E2EE key rotation are deferred.** A session-setup race can
>   sometimes deliver zero frames on connect. If that occurs, retry the connect.
>   Pin a version and test your flows.

```diff
- import { Room, RoomEvent, createLocalAudioTrack } from "livekit-client";
+ import { Room, RoomEvent, createLocalAudioTrack } from "@clutchcall/livekit-compat";
```

```ts
const room = new Room();
await room.connect(url, token);              // to your relay.firstbyte.fast endpoint
await room.localParticipant.setMicrophoneEnabled(true);
room.on(RoomEvent.TrackSubscribed, (track) => track.attach());
```

The mic audio still goes through the browser's Opus encoder. But the shim
extracts only the **encoded frames** and sends them over MoQT. The shim
requires encoded media transforms (`RTCRtpScriptTransform`). It **refuses to
connect** on a browser that would silently fall back to plaintext WebRTC. Today
that means Chromium-family browsers. The full deep-dive is in
[Keep Your Existing Runtime](/modalities/voice/runtime/keep-existing) and
[WebRTC Diversion](/modalities/voice/transport-web/webrtc-diversion).

## Which path?

| You have | Use | Status |
| --- | --- | --- |
| A LiveKit **agent**, and you want to drop the SFU entirely | The transport plugin (Path A) | Shipped |
| A LiveKit **agent** whose room must keep running | Front-proxy the media (Path B) | Shipped edge + a sidecar you run |
| A LiveKit **browser app** (`livekit-client`) | The `livekit-compat` shim (Path C) | Preview |
| A LiveKit agent on **serverless** (scale-to-zero) | Front-proxy (Path B) until per-call trigger lands | Plugin is register-and-wait today |

## Related

  - **[Run a LiveKit Agent on the Transport](/modalities/voice/recipes/livekit-transport)** — The end-to-end recipe: install the plugin, register a worker, and take a call.
  - **[Keep Your Existing Runtime](/modalities/voice/runtime/keep-existing)** — The full transport-only spectrum: the plugin, the shim, and raw MoQT.
  - **[From Self-Hosted LiveKit](/modalities/voice/migration/from-livekit)** — The end-to-end migration path: trunks, tokens, and rollout.
  - **[Custom Agent Runtime](/modalities/voice/integrations/custom-runtime)** — The same patterns for any non-LiveKit runtime.
  - **[Outbound Calls from a LiveKit Agent](/modalities/voice/recipes/livekit-outbound)** — Dial one number or run a paced campaign. The same worker answers.
  - **[Embed an Agent Widget](/modalities/voice/recipes/web-embed)** — Put the same agent on your website. One tag plus one backend route.
  - **[Recordings API](/modalities/voice/api/recordings)** — List recordings, presign a download, or get pushed a webhook.