# OpenAI-Compatible HTTP API

> Call deployed clutch-layers models over the OpenAI-compatible HTTP surface: base URL, endpoints, SSE streaming over HTTP/2 and HTTP/3, and tool calling.

Deployed inference models are reachable over an OpenAI-compatible HTTP
API. Any client that speaks the OpenAI REST shape — the official SDKs,
`curl`, or your own HTTP client — works against it after you change the
base URL and the API key.

This page covers that HTTP surface. It is separate from the
speech-to-speech agent path, where a model is attached to a call through
the Voice surface instead of being called directly.

## Base URL and authentication

There is a single, operator-global base URL. Per-tenant URL routing is
not implemented in `mod_inference` today, so every tenant sends requests
to the same host and the **API key identifies the caller**. The console's
**API keys → Base URL** card shows the exact value and has a copy
button next to it; treat that card as the source of truth rather than
hardcoding a URL from documentation.

Store the value alongside your key:

```bash
export INFERENCE_BASE_URL=…        # copy from API keys → Base URL
export CLUTCHCALL_CREDENTIALS=…       # your FirstByte API key
```

The key travels as a bearer token:

```
Authorization: Bearer $​CLUTCHCALL_CREDENTIALS
Content-Type: application/json
```

Because the key selects the tenant, it also selects which deployed
models you can address. A `model` id that is not deployed for the
tenant behind the key is not addressable.

## Endpoint reference

Five OpenAI-compatible paths are exposed under the base URL. The gateway serves
HTTP/2 and HTTP/3 only. It does not serve HTTP/1.1.

| Path                | Purpose                                                                 |
| ------------------- | ----------------------------------------------------------------------- |
| `/chat/completions` | Chat-style generation. Supports `stream: true` and the `tools` field.   |
| `/audio/transcriptions` | Speech-to-text.                                                      |
| `/audio/speech`     | Text-to-speech.                                                          |
| `/models`           | Lists the model ids addressable with the presented API key.              |
| `/health`           | Liveness of the inference surface. Use it for probes and uptime checks.  |

The `model` field in every request body takes a deployed **model id**.
Read the ids from `/models` (or from the console's model list) rather
than guessing them — the snippets in the console ship with a
`<model-id>` placeholder for exactly this reason.

A minimal non-streaming chat request:

```bash
curl $INFERENCE_BASE_URL/chat/completions \
  -H "Authorization: Bearer $​CLUTCHCALL_CREDENTIALS" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id>",
    "messages": [{ "role": "user", "content": "Hello" }]
  }'
```

## Streaming over HTTP/3

Set `stream: true` and the response is delivered as Server-Sent Events.
The gateway carries them over HTTP/3 or HTTP/2. Each event is an OpenAI-shaped chunk, so
`chunk.choices[0].delta.content` is where incremental text arrives.

Two consequences worth knowing:

- **No client changes are required.** The OpenAI SDKs iterate the SSE
  stream exactly as they do against any other OpenAI-compatible server.
  The HTTP client negotiates HTTP/2 or HTTP/3 underneath. A client that can
  use only HTTP/1.1 cannot connect.
- **0-RTT resumption cuts reconnect latency (HTTP/3).** When a client reconnects
  to a host it has already talked to, the QUIC session can resume
  without a full handshake round trip.

```bash
curl $INFERENCE_BASE_URL/chat/completions \
  -H "Authorization: Bearer $​CLUTCHCALL_CREDENTIALS" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id>",
    "stream": true,
    "messages": [{ "role": "user", "content": "Hello" }]
  }'
```

## Tool calling

Tool and function calling use the OpenAI `tools` field. The model server
decides whether a model supports it. The request cannot turn tool calling on
for a model whose server does not support it. The deploy spec of the model
(see the [model registry](/modalities/inference/model-registry)) has no tool
flags. When the model supports tool calling, the `tools` field works
unchanged:

```ts
const res = await client.chat.completions.create({
  model: "<model-id>",
  messages: [{ role: "user", content: "What is the weather in Lisbon?" }],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        parameters: {
          type: "object",
          properties: { city: { type: "string" } },
          required: ["city"],
        },
      },
    },
  ],
});
```

If tool calls never come back for a model that you expect to support them,
test the model with a small request and no client code. A model that does not
support tools gives a plain text answer.

## Using the official OpenAI SDKs

Point the SDK's base URL at the inference base URL and pass your
FirstByte API key. Nothing else changes. These are the snippets the
console's **SDK quickstart** card emits.

**Python**

```python
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["INFERENCE_BASE_URL"],
    api_key=os.environ["CLUTCHCALL_CREDENTIALS"],
)

stream = client.chat.completions.create(
    model="<model-id>",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")
```

**Node**

```ts
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: process.env.INFERENCE_BASE_URL,
  apiKey: process.env.CLUTCHCALL_CREDENTIALS,
});

const stream = await client.chat.completions.create({
  model: "<model-id>",
  messages: [{ role: "user", content: "Hello" }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
```

Keep the API key on your backend. It is a long-lived, tenant-scoped
credential; see [Authentication](/concepts/authentication) for the
pattern to use when a browser needs access.

## Related

- [Authentication](/concepts/authentication) — API keys and how they scope a caller
- [Modalities overview](/modalities/overview)