# Agents

> Attach a server-side voice agent to a live call. The AgentSpec defines the agent's mode, providers, turn detection, and tools.

An **agent** is the AI voice pipeline that answers a call. The pipeline defines
the mode (cascade or duplex), the provider for each stage, when the agent yields
the floor, and the tools the agent can call. You author the pipeline once as an
**AgentSpec**. You then address the agent by id. Use the Agents API to bind an
agent to a live call. One call to `agents.attach(sid, agentId)` connects the
audio bridge end to end.

This is the "drop-in voice agent" path. You do not open an
[AudioBridge](/modalities/voice/api/sessions). You do not move audio frames
yourself. You give the runtime a `call_sid` and an agent id. The runtime then
controls the call.

> **NOTE:**
> The SDK attach call takes an **agent id string**, not an inline spec object. The
> AgentSpec stays on the server. Author the spec in the console at
> `agent.firstbyte.fast` or push it through the control-plane API. Then reference
> the spec by id from every call. This keeps provider keys and prompts out of your
> client code. You can also re-tune an agent without a client release. This page
> documents the spec fields in summary.
> [Runtime configuration](/modalities/voice/runtime/configuration) documents the
> fields in full.

## Attach an agent to a call

Bind an agent to a `call_sid`. You get the `call_sid` from
`voice.calls.originate`, from an inbound-call webhook, or from
`voice.calls.get`. The call must be live (ringing or in progress).

  
    ```ts
    import { Voice } from "@clutchcall/agents";

    const voice = new Voice({
      baseUrl: "https://engine.firstbyte.fast",
      apiKey:  process.env.CLUTCHCALL_API_KEY!,
      orgId:   "<workspace-id>",
    });

    // Bind the server-side agent "support-agent" to a live call.
    await voice.agents.attach(callSid, "support-agent");
    ```
  
  
    ```python
    from clutchcall.voice import Voice

    voice = Voice(
        base_url="https://engine.firstbyte.fast",
        api_key=os.environ["CLUTCHCALL_API_KEY"],
        org_id="<workspace-id>",
    )

    # Bind the server-side agent "support-agent" to a live call.
    voice.agents.attach(call_sid, "support-agent")
    ```
  

`attach` resolves the control-plane route `voice.agents.attach` with your
`orgId`, the `sid`, and the `agent` id. On success, the engine hydrated the
AgentSpec, prewarmed the providers, and connected the caller's audio to the
pipeline. The same signature exists in the Go and Rust SDKs
(`Agents.Attach(callSid, agent)` / `agents.attach(call_sid, agent)`). See the
[SDK reference](/modalities/voice/sdk/javascript).

### Parameters

- **``** (`string`, required) — The `call_sid` of a live call. This is the only identifier that you need to   address a call. The SIP dialog, the media session, the agent session, and the   telemetry all key off this id.

- **``** (`string`, required) — The id of a server-side agent (its AgentSpec). Author the agent in the console   at `agent.firstbyte.fast` or through the control-plane API. Do this before you   reference the agent.

### Attach at originate time

For outbound calls, you rarely need a separate attach. Pass the agent id to
`originate`. The engine binds the agent automatically on answer. The agent
speaks as soon as the callee answers.

  
    ```ts
    const call = await voice.calls.originate({
      to:      "+15551234567",
      from:    "+15559876543",
      trunkId: "<trunk-id>",
      agent:   "support-agent",   // attached on answer — no second call
    });
    ```
  
  
    ```python
    call = voice.calls.originate(
        to="+15551234567",
        from_="+15559876543",
        trunk_id="<trunk-id>",
        agent="support-agent",   # attached on answer — no second call
    )
    ```
  

For **inbound** calls, your dialplan or trunk routing selects the agent (a DID
or a skill maps to an agent). No client attach is necessary. Use
`agents.attach` when you want to bind or swap the agent from your own code.
Examples: after you screen a call, on an IVR selection, or in a webhook handler.

### Re-attach to a different agent

A swap of agents mid-call is a **transfer to an agent**, not a second attach.
The transfer stops the current pipeline cleanly before the new pipeline takes
over. Use `Call.transfer` with an `agent` (see
[Calls](/modalities/voice/api/calls)):

```ts
await call.transfer({ agent: "billing-agent" });
```

A handoff of a live call from an AI agent to a **human** is a different
operation. See [AI ↔ human handoffs](/modalities/voice/concepts/handoffs).

## The AgentSpec

The AgentSpec is the single document that defines an agent. `attach` binds a
call to the spec by id. This section lists the fields that the spec resolves.
Every field has a default. A minimal spec is a name, a prompt, and the providers
that you want to override. This section is a summary of the top-level shape.
[Runtime configuration](/modalities/voice/runtime/configuration) is the full
field-by-field reference, with every timing knob and budget.

- **``** (`string`, required) — The human-readable agent name. The console shows this name. The engine stamps   this name onto call records, so you can tell agents apart in the dashboards.

- **``** (`string`, default `cascaded`) — The agent **mode**. `cascaded` chains three provider sessions   (speech-to-text → language model → text-to-speech). This mode gives per-stage   control and the widest model choice. `realtime` gives the full turn to one   duplex speech-to-speech model. This mode gives the lowest turn latency. See   [BYO ASR/LLM/TTS](/modalities/voice/runtime/byo-asr-llm-tts) and   [BYO speech-to-speech](/modalities/voice/runtime/byo-speech-to-speech).

- **``** (`string`) — The agent's persona and task (`instructions` on a realtime node). Keep replies   short and spoken-friendly. Short sentences work better than paragraphs on a   phone line.

### Model & provider

The provider fields that you set depend on the mode. Provider **keys** are
per-tenant. The control plane resolves the keys at call setup. The spec never
contains the keys. If a selected provider has no usable key, the engine refuses
the agent at session creation with a clean error. The agent does not fail
mid-call.

- **``** (`string`) — Cascade only. The streaming speech-to-text vendor and model — for example,   `deepgram` / `nova-2`. The engine feeds the vendor at the runtime's 8 kHz   PCM16 contract. The engine resamples for providers that natively want   16/24 kHz.

- **``** (`string`) — Cascade only. The language-model vendor and model — `openai`, `anthropic`,   `gemini`, or self-hosted `ollama`, with a model id such as `gpt-4o-mini` or   `claude-sonnet-4-5`. Tool calling runs at this stage. Set   `llm_fallback_provider` for a one-turn resilience path.

- **``** (`string`) — Cascade only. The text-to-speech vendor and voice id — for example,   `elevenlabs`, `cartesia`, `deepgram`, or `openai`, with the provider's voice   id.

- **``** (`string`) — Realtime only. On a `realtime` entry node, you set one duplex provider instead   of three cascade stages. Examples: `openai` / `gpt-realtime` / `shimmer`, or   `gemini` with a `Kore` voice. One model listens and speaks on the same   bidirectional session.

> **NOTE:**
> Shipped duplex providers include OpenAI Realtime, Gemini Live, and xAI Grok
> Voice. Support to serve your **own** open-weights duplex model on your GPUs is
> in preview. See
> [self-hosted inference](/modalities/voice/runtime/self-hosted-inference).

### turn_detection

This block controls how the agent decides that the caller stopped talking. It
also controls how the agent yields the floor when the caller barges in. Omit
the block for phone-safe defaults.
[Turn Detection & Barge-in](/modalities/voice/concepts/turn-detection)
documents the full behavior.

- **``** (`string`, default `on-device VAD`) — This field sets which component runs voice-activity detection. The default   (omit `type`) runs an **on-device VAD** on the gateway. The default is the   correct choice for cascaded agents. Set `"energy"` for the lightweight   built-in energy gate. `"server_vad"` defers end-of-turn to a cloud realtime   provider that already emits speech-start/stop events. The runtime auto-selects   `"server_vad"` for a `realtime` entry node unless you set `type` explicitly.   A cascade with `server_vad` never sees a turn boundary. Use the on-device   default for a cascade.

- **``** (`integer`, default `500`) — The trailing silence before end-of-utterance fires. Set a lower value (~300)   for faster replies. Set a higher value (~800) for callers who pause to think.

- **``** (`integer`, default `300`) — The **hold-and-confirm** barge-in window. A speech onset arms a *pending*   barge. The pending barge cancels the agent only if speech continues past this   window. As a result, a backchannel ("mhm", "ok") never cuts the agent off.   Set `0` for legacy immediate barge-in. Use `0` only on echo-cancelled   (browser) transports.

- **``** (`object`) — The idle watchdog. `soft_prompt_s` re-engages a silent caller. `hangup_s` then   ends the call. The watchdog is on by default, with sensible ceilings, so a   stuck agent cannot hold a line open forever.

### tools

The tools that the agent can call mid-conversation: an HTTP endpoint, a remote
MCP server, or a handler in your own SDK. The
[Agent DAG](/modalities/voice/runtime/overview) uses the same tool model. An
`llm` (or `realtime`) node advertises tools by name. The model decides when to
call a tool. The runtime validates the arguments against the tool's JSON Schema
before dispatch. Each session also unions your tools with a built-in
**telephony toolset** (transfer, hold, DTMF, disconnect, supervisor request).
On a name collision, your tool wins.
[Tool calling](/modalities/voice/runtime/tool-calling) is the full reference.

- **``** (`array`) — A list of tool definitions. Each entry shares one envelope: `kind`, `name`,   `description`, `silent`, and `spec`. The kind-specific configuration goes   inside `spec`.

- **``** (`string`, required) — `http` (shipped) calls an endpoint that you own. `mcp` (shipped) runs a   JSON-RPC `tools/call` against a remote MCP server over Streamable HTTP.   `client` (scaffolded) pushes the call to your connected SDK, which fulfils   the call.

- **``** (`string`, required) — The function name that the model sees and calls, and the human-readable   purpose that the model reads. Write the description for the model. The   description is the primary signal for *when* the model calls the tool.

- **``** (`object`) — A standard **JSON Schema** for the tool's arguments. On an `http` tool, you   can omit the schema. The runtime then synthesizes a schema from the   `{{placeholder}}` tokens in your URL and body.

### Example AgentSpec

This example shows a cascaded support agent with one HTTP tool. Author the spec
in the console, or push the spec through the control-plane API. Then reference
the spec by id (`support-agent`) from `agents.attach`.

```json AgentSpec
{
  "name": "support-agent",
  "pipeline_mode": "cascaded",
  "system_prompt": "You are a friendly support agent for Acme. Keep replies short and spoken-friendly.",
  "asr_provider": "deepgram",
  "asr_model": "nova-2",
  "llm_provider": "openai",
  "llm_model": "gpt-4o-mini",
  "llm_fallback_provider": "anthropic",
  "tts_provider": "elevenlabs",
  "tts_voice": "<voice-id>",
  "turn_detection": {
    "silence_threshold_ms": 500,
    "barge_confirm_ms": 300
  },
  "tools": [
    {
      "kind": "http",
      "name": "lookup_order",
      "description": "Look up an order by its ID and return its status.",
      "spec": {
        "method": "GET",
        "url": "https://api.example.com/orders/{{order_id}}",
        "parameters": {
          "type": "object",
          "properties": { "order_id": { "type": "string" } },
          "required": ["order_id"]
        }
      }
    }
  ]
}
```

## Related

  - **[Runtime configuration](/modalities/voice/runtime/configuration)** — The full AgentSpec field reference, with every provider, timing, and budget knob.
  - **[Calls](/modalities/voice/api/calls)** — Originate, transfer (re-attach to a different agent), and hang up.
  - **[Tool calling](/modalities/voice/runtime/tool-calling)** — Give the agent HTTP, MCP, and client tools that it can call mid-conversation.
  - **[Turn detection & barge-in](/modalities/voice/concepts/turn-detection)** — The behavior behind the turn_detection block.
  - **[Sessions & audio bridge](/modalities/voice/api/sessions)** — The lower-level path, for when you want to move audio frames yourself.
  - **[AI ↔ human handoffs](/modalities/voice/concepts/handoffs)** — Hand a live call from the agent to a human.