# Pipecat Agent with FirstByte Transport

> Build an end-to-end app that answers real calls with a Pipecat pipeline over QUIC/MoQT. It uses the shipping audio-bridge spine today, and shows the shape of a first-class transport.

You have a Pipecat bot: a pipeline of frame processors wired to ASR, an LLM,
and TTS. You want it to answer real phone calls and browser sessions over
FirstByte's transport. You do not want to operate a media server or
negotiate ICE/DTLS/SRTP. This recipe assembles that end to end. You build a
small host process. The process takes a live call, subscribes to the caller's
audio, runs the audio through a Pipecat pipeline, and streams the agent's
replies back. All of this runs over QUIC/MoQT on the single `:443` plane.

> **WARNING:**
> **A packaged Pipecat adapter now ships**: `pipecat-clutchcall`. It is a
>   drop-in `ClutchCallTransport` over the external-agent QUIC plane, with one
>   pipeline per routed call. Start from
>   [Pipecat + FirstByte Transport](/modalities/voice/integrations/pipecat)
>   unless you specifically want this recipe's lower-level path. That path is
>   the **audio bridge**: caller audio in, agent audio out, over raw MoQT. On
>   that path, you own the thin glue that hands frames to Pipecat. The
>   `QuicMoqtTransport` wrapper near the end remains an *illustrative sketch* of
>   that hand-rolled shape, not the shipped package.

## What you're building

Pipecat and FirstByte sit at different layers. That is why they compose
cleanly. **Pipecat owns the conversation**: the frame pipeline, turn-taking,
and your model choices. **FirstByte owns the transport and telephony**: the
SIP gateway/B2BUA that terminates the carrier trunk, and the media plane that
carries the audio over QUIC. The seam between them is a pair of tracks keyed by
the call id: an **uplink** (caller → bot) and a **downlink** (bot → caller):

```text
  caller ──SIP/RTP──▶ SIP gateway / B2BUA ──▶ QUIC/MoQT media plane
                                                     │
                             voice/<call_sid>/uplink │ (caller audio, PCM16)
                                                     ▼
   ┌─────────────────────── your host process ───────────────────────┐
   │  AudioBridge.onUplink ─▶ Pipecat pipeline ─▶ AudioBridge.publish │
   │     (SDK, real)          (STT → LLM → TTS)     Downlink (SDK)     │
   └──────────────────────────────────────────────────────────────────┘
                                                     ▲
                           voice/<call_sid>/downlink │ (agent audio, PCM16)
```

The two `AudioBridge` calls are stock SDK. Everything between them is your
Pipecat pipeline. The pipeline is unchanged from any other Pipecat deployment.

## Build the app

  1. **Take the call and attach the audio bridge**
Skip `agents.attach` (that call runs *our* managed runtime). Open the
    bridge yourself. Ask for `pcm16` at 16 kHz. That is the shape most frame
    pipelines expect. The bridge transcodes the PSTN (G.711) leg and the
    runtime's 8 kHz bus for you in both directions.

    
      
```ts
import { Voice } from "clutchcall_connect/voice";

const v = new Voice({
  baseUrl: "https://engine.firstbyte.fast",
  apiKey:  process.env.CLUTCHCALL_CREDENTIALS!,
  orgId:   "org_abc",
});

// Originate (or attach to an inbound sid you already hold).
const call = await v.calls.originate({
  to: "+15551234567",
  from: "+15558675309",
  trunkId: "trunk_main",
});

// pcm16 @ 16 kHz — raw audio in both directions.
const bridge = await v.audioBridge.attach(call.sid, {
  codec: "pcm16",
  sampleRate: 16000,
  onUplink: (pcm, tsUs) => pipeline.pushAudio(pcm),      // caller -> bot
});
```
      
      
```python
import os
from clutchcall.voice import Voice

v = Voice(
    base_url="https://engine.firstbyte.fast",
    api_key=os.environ["CLUTCHCALL_CREDENTIALS"],
    org_id="org_abc",
)

# Originate (or attach to an inbound sid you already hold).
call = v.calls.originate(
    to="+15551234567", from_="+15558675309", trunk_id="trunk_main",
)

# pcm16 @ 16 kHz — raw audio in both directions.
bridge = v.audio_bridge.attach(
    call.sid,
    codec="pcm16",
    sample_rate=16000,
    on_uplink=lambda pcm, ts_us: pipeline.push_audio(pcm),   # caller -> bot
)
```
      
    

  2. **Wrap the two hooks as a Pipecat transport**
Pipecat's `Transport` seam needs exactly two things: a producer of input
    audio frames and a consumer of output audio frames. The bridge already
    gives you both. `onUplink` is the producer. `publishDownlink` is the
    consumer. Wrap them so that the rest of the pipeline runs unchanged.

    > **NOTE:**
> **Illustrative — you own this class today.** The transport below is
>       *your* glue code around the real audio bridge, not an installable
>       package. Class and frame names track Pipecat's public shape. When you
>       write this class for real, pin your Pipecat version and match its exact
>       transport base classes.

```python
# ILLUSTRATIVE glue you write today — wraps the REAL audio bridge for Pipecat.
from pipecat.frames.frames import InputAudioRawFrame, OutputAudioRawFrame
from pipecat.processors.frame_processor import FrameProcessor

class MoqtBridgeInput(FrameProcessor):
    """Emits caller uplink audio as Pipecat input frames."""
    def __init__(self, sample_rate: int = 16000):
        super().__init__()
        self.sample_rate = sample_rate

    def on_uplink(self, pcm: bytes, ts_us: int):
        frame = InputAudioRawFrame(audio=pcm, sample_rate=self.sample_rate, num_channels=1)
        # hand the frame downstream into the pipeline
        self.push_frame(frame)

class MoqtBridgeOutput(FrameProcessor):
    """Publishes agent output frames back as downlink audio."""
    def __init__(self, bridge):
        super().__init__()
        self.bridge = bridge

    async def process_frame(self, frame, direction):
        await super().process_frame(frame, direction)
        if isinstance(frame, OutputAudioRawFrame):
            self.bridge.publish_downlink(frame.audio)   # bot -> caller (SDK, real)
        await self.push_frame(frame, direction)
```

  3. **Assemble the pipeline**
Drop the two wrappers on the ends of an ordinary Pipecat pipeline. The STT,
    LLM, and TTS processors are stock Pipecat. Pick the providers that you
    already use.

```python
# ILLUSTRATIVE — a standard Pipecat pipeline between the two bridge wrappers.
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask

bridge_in  = MoqtBridgeInput(sample_rate=16000)
bridge_out = MoqtBridgeOutput(bridge)

# route the SDK's uplink callback into the input processor
# (v.audio_bridge.attach(..., on_uplink=bridge_in.on_uplink))

pipeline = Pipeline([
    bridge_in,     # caller audio in
    stt,           # your ASR frame processor
    llm,           # your LLM context
    tts,           # your TTS frame processor
    bridge_out,    # agent audio back to the caller
])
```

  4. **Run the call, then tear down cleanly**
Run the pipeline for the life of the call. When the conversation ends,
    close the bridge **before** you hang up. Both tracks then tear down
    cleanly.

    
      
```ts
// … your pipeline runs; when it decides the call is over:
await bridge.close();     // release both tracks first
await call.hangup();
```
      
      
```python
runner = PipelineRunner()
try:
    await runner.run(PipelineTask(pipeline))
finally:
    bridge.close()        # release both tracks first
    call.hangup()
```
      
    

    > **WARNING:**
> Keep the `bridge` reference alive for the whole call. If the garbage
>       collector removes it mid-call, both tracks drop and the audio goes
>       silent.

## Where this is headed

A first-class integration would collapse the two wrapper classes above into a
single Pipecat `Transport`. That transport would bind directly to our
media-over-QUIC primitives. You would construct it with the call id. You would
drop `transport.input()` / `transport.output()` onto the pipeline. You would
write no `onUplink`/`publishDownlink` glue.

> **NOTE:**
> **Design sketch — not shipped.** `QuicMoqtTransport` does not exist as a
>   package. It shows the intended shape of a future adapter only.

```python
# ILLUSTRATIVE — this transport is not shipped. Design sketch only.
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask

# A transport that would carry Pipecat frames over the QUIC/MoQT plane:
#   input():  subscribe voice/<call_sid>/uplink   -> InputAudioRawFrame
#   output(): publish   voice/<call_sid>/downlink  <- OutputAudioRawFrame
transport = QuicMoqtTransport(
    url="quic://relay.firstbyte.fast",
    call_sid=call.sid,
    sample_rate=16000,
)

pipeline = Pipeline([
    transport.input(),   # caller audio in
    stt, llm, tts,       # your pipeline, unchanged
    transport.output(),  # agent audio back to the caller
])

await PipelineRunner().run(PipelineTask(pipeline))
```

The only real work that such an adapter adds is the seam that our own runtime
already handles. The adapter must do three things:

- resample between Pipecat's frame rate and the runtime's **8 kHz PCM16**
  audio bus,
- keep the room-scoped MoQT namespaces valid, and
- carry **Opus** rather than PCM on browser legs.

None of this is exotic. It is not packaged yet, so today you own the wrapper.

## Other on-ramps that ship today

If you do not want to write even the wrapper, two seams already let a Pipecat
bot take live calls:

  
    Pipecat already speaks WebSocket audio. FirstByte accepts a bespoke
    WebSocket audio sender as a media leg. Point Pipecat's WebSocket server
    transport at that on-ramp. Route the trunk's inbound rule to your endpoint.
    The SIP gateway terminates the carrier while Pipecat drives the
    conversation, with **no FirstByte code in the bot**. This is the same
    path as
    [migrating a custom WebSocket audio sender](/modalities/voice/migration/from-websocket-audio).
  
  
    Your Pipecat bot can front a LiveKit, Vapi, or Twilio media backend. In
    that case, the config-only
    [VENDOR_BRIDGE](/modalities/voice/runtime/keep-existing) node routes call
    media to that backend via a sidecar. Pipecat itself is **not** a bridge
    target. This seam helps only when one of those vendors sits behind Pipecat.
  

> **TIP:**
> You can want Pipecat only for the *pipeline*: cascaded ASR → LLM → TTS,
>   turn-taking, tools. FirstByte's runtime already ships that pipeline as a
>   configured DAG with transport, telephony, and barge-in built in. You can
>   bring only your **models** via provider credentials. That is often less work
>   than a port of a Pipecat pipeline onto an adapter that does not exist yet.

## Related

  - **[Pipecat + Transport](/modalities/voice/integrations/pipecat)** — The full conceptual framing and the on-ramps that ship today.
  - **[Bridge a Pipecat Agent](/modalities/voice/cookbooks/bridge-pipecat)** — The single-snippet version of the audio-bridge seam that this recipe uses.
  - **[LiveKit Agent with FirstByte Transport](/modalities/voice/recipes/livekit-transport)** — The sibling recipe for a LiveKit agent over the same transport.
  - **[Keep Your Existing Runtime](/modalities/voice/runtime/keep-existing)** — Adopt only the transport and keep your own agent loop.