# Gemini Live

> Run Google's Gemini Live as the duplex speech-to-speech model on a call. You configure it fully from the agent spec.

Point an agent at **Gemini Live**, and the caller talks to Google's duplex
speech-to-speech model directly. There is no separate transcriber or
text-to-speech stage. The caller's audio streams into one bidirectional
session. The session listens and speaks on the same connection. Thus the agent
starts to reply sooner, and barge-in feels instant.

This is a **shipped, first-class provider**. FirstByte Voice ships a native
Gemini Live adapter. The adapter speaks Google's `BidiGenerateContent`
WebSocket protocol end to end. The adapter does these tasks:

- It opens the session.
- It upsamples the call's audio to the rate that Gemini expects.
- It streams the model's spoken reply back onto the leg.
- It connects Gemini's server-side voice activity detection to barge-in.

To turn it on, name `gemini` as the provider in your agent config. There is no
code to write.

> **NOTE:**
> Gemini Live is one of several duplex providers. For the shared model of how a
> speech-to-speech agent works, see
> [BYO Speech-to-Speech Models](/modalities/voice/runtime/byo-speech-to-speech).
> That page covers the single `REALTIME` node, the greeting behavior, and
> session budgets. This page covers the Gemini-specific config, voices, and
> behavior.

## Configure a Gemini Live agent

A duplex agent has a single node. Set the entry node to `REALTIME`. Name
`gemini` as the provider. Select a model and a voice.

  1. **Name gemini in the agent spec**
**agent-config.yml:**
```yaml
    entry_node: REALTIME
    pipeline:
      REALTIME:
        node_type: REALTIME
        provider: gemini
        model: gemini-2.0-flash-exp    # "models/" prefix added for you if omitted
        voice: Aoede                   # Aoede | Charon | Fenrir | Kore | Puck
        instructions: |
          You are a friendly support agent for Acme. Keep replies short and
          speak naturally.
    ```
    

    - **`provider`** (`string`, required) — Set to `gemini` to select the Gemini Live adapter.
    - **`model`** (`string`) — A Gemini Live model id, for example `gemini-2.0-flash-exp`. The adapter       qualifies the id to Gemini's required `models/<id>` form for you. Thus       `gemini-2.0-flash-exp` and `models/gemini-2.0-flash-exp` both work.
    - **`voice`** (`string`) — One of Gemini's prebuilt voices: `Aoede`, `Charon`, `Fenrir`, `Kore`, or       `Puck`. Leave the field empty to accept Google's server default. The       runtime defaults to `Aoede` when you omit the field.
    - **`instructions`** (`string`) — The system prompt. The adapter sends it as Gemini's `systemInstruction`       in the opening setup message. It also seeds the inbound greeting.

  2. **Supply your Gemini API key**
The system stores credentials **per tenant** and never shares them across
    workspaces. Add your Gemini API key in the Voice AI console at `voiceai.firstbyte.fast`, or through the control-plane API at
    `portal.firstbyte.fast`. The runtime reads the key when it opens the
    session. If there is no key, the provider returns no session. The call then
    falls back to a silent passthrough and does not drop.

  3. **Point a number at the agent and call**
Save the agent. Attach a phone number or trunk to it. Place a call.
    FirstByte Voice opens the Gemini Live WebSocket on the first inbound
    audio frame. It then streams audio in both directions.

## How the audio flows

You do not touch sample rates. The adapter does the conversions that Gemini
requires:

  1. **Caller audio in**
The adapter upsamples the call's 8&nbsp;kHz PCM16 to the **16&nbsp;kHz
    PCM16** that Gemini Live expects on input. It streams the audio as
    `realtimeInput.audio` frames.
  2. **Model audio out**
Gemini returns **24&nbsp;kHz PCM16** audio. The runtime resamples the audio
    down and paces it back onto the call leg. A phone leg gets G.711. A browser
    leg gets Opus.
  3. **Barge-in**
When the caller talks over the agent, Gemini's server VAD emits an
    `interrupted` signal. The runtime cuts the in-flight reply immediately.

## Turn-taking

For a `REALTIME` entry node, the runtime defaults to **`server_vad`**. Gemini
Live does its own endpointing and interruption detection. Thus you do **not**
add a local turn detector. When Gemini reports an interruption, the runtime
maps it to barge-in and stops the outgoing audio. See
[Turn Detection & Barge-In](/modalities/voice/concepts/turn-detection) to tune
how quickly the agent yields, or to learn the fallback behavior for models
without server VAD.

## Greeting on inbound calls

Some carrier SBCs withhold RTP until they hear your RTP. On an inbound call,
the runtime prompts Gemini to speak first. This opens the media path. The
model greets from your `instructions`. This behavior is on by default for
inbound direct-media calls. No extra config is required.

## Tools mid-conversation

The Gemini Live adapter renders your configured tools into Gemini's
`functionDeclarations` in the opening setup message. It receives `toolCall`
events during a spoken turn. It returns your result as a `toolResponse`. Thus
the model can call an HTTP or MCP tool and speak the outcome without leaving
the call. Configure tools in the same way as for any agent. See
[Tool Calling](/modalities/voice/runtime/tool-calling).

> **NOTE:**
> Function calling over Gemini Live is newer than the
> [OpenAI Realtime](/modalities/voice/integrations/openai-realtime) path. OpenAI
> Realtime is the most tested realtime adapter today. If your agent depends on
> complex multi-tool turns, check the behavior on a staging number before you
> route production traffic.

## Session budgets

Every realtime session is bounded. A stuck provider cannot run forever. The
handshake gets a few seconds. After that, the runtime gives up and falls back.
The runtime closes sessions at a **15-minute** ceiling, even if the provider
did not close them. Tune lifetime, idle re-engage, and hangup budgets on
[Session Lifecycle](/modalities/voice/runtime/session-lifecycle).

## Next steps

  - **[Speech-to-speech model](/modalities/voice/runtime/byo-speech-to-speech)** — The shared duplex-agent model, greeting behavior, and provider matrix.
  - **[OpenAI Realtime](/modalities/voice/integrations/openai-realtime)** — The most complete realtime adapter: server VAD, function calling, and greeting.
  - **[Turn detection](/modalities/voice/concepts/turn-detection)** — Tune barge-in, endpointing, and backchannel handling.
  - **[All runtime knobs](/modalities/voice/runtime/configuration)** — Models, codecs, budgets, and guardrails in one reference.
