OpenAI-Compatible HTTP API

Deployed inference models are reachable over an OpenAI-compatible HTTP API. Any client that speaks the OpenAI REST shape — the official SDKs, curl, or your own HTTP client — works against it after you change the base URL and the API key. This page covers that HTTP surface. It is separate from the speech-to-speech agent path, where a model is attached to a call through the Voice surface instead of being called directly.

Base URL and authentication

There is a single, operator-global base URL. Per-tenant URL routing is not implemented in mod_inference today, so every tenant sends requests to the same host and the API key identifies the caller. The console’s API keys → Base URL card shows the exact value and has a copy button next to it; treat that card as the source of truth rather than hardcoding a URL from documentation. Store the value alongside your key:

export INFERENCE_BASE_URL=…        # copy from API keys → Base URL
export CLUTCHCALL_CREDENTIALS=…       # your FirstByte API key

The key travels as a bearer token:

Authorization: Bearer $​CLUTCHCALL_CREDENTIALS
Content-Type: application/json

Because the key selects the tenant, it also selects which deployed models you can address. A model id that is not deployed for the tenant behind the key is not addressable.

Endpoint reference

Five OpenAI-compatible paths are exposed under the base URL. The gateway serves HTTP/2 and HTTP/3 only. It does not serve HTTP/1.1.

Path	Purpose
`/chat/completions`	Chat-style generation. Supports `stream: true` and the `tools` field.
`/audio/transcriptions`	Speech-to-text.
`/audio/speech`	Text-to-speech.
`/models`	Lists the model ids addressable with the presented API key.
`/health`	Liveness of the inference surface. Use it for probes and uptime checks.

The model field in every request body takes a deployed model id. Read the ids from /models (or from the console’s model list) rather than guessing them — the snippets in the console ship with a <model-id> placeholder for exactly this reason. A minimal non-streaming chat request:

curl $INFERENCE_BASE_URL/chat/completions \
  -H "Authorization: Bearer $​CLUTCHCALL_CREDENTIALS" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id>",
    "messages": [{ "role": "user", "content": "Hello" }]
  }'

Streaming over HTTP/3

Set stream: true and the response is delivered as Server-Sent Events. The gateway carries them over HTTP/3 or HTTP/2. Each event is an OpenAI-shaped chunk, so chunk.choices[0].delta.content is where incremental text arrives. Two consequences worth knowing:

No client changes are required. The OpenAI SDKs iterate the SSE stream exactly as they do against any other OpenAI-compatible server. The HTTP client negotiates HTTP/2 or HTTP/3 underneath. A client that can use only HTTP/1.1 cannot connect.
0-RTT resumption cuts reconnect latency (HTTP/3). When a client reconnects to a host it has already talked to, the QUIC session can resume without a full handshake round trip.

curl $INFERENCE_BASE_URL/chat/completions \
  -H "Authorization: Bearer $​CLUTCHCALL_CREDENTIALS" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id>",
    "stream": true,
    "messages": [{ "role": "user", "content": "Hello" }]
  }'

Tool calling

Tool and function calling use the OpenAI tools field. The model server decides whether a model supports it. The request cannot turn tool calling on for a model whose server does not support it. The deploy spec of the model (see the model registry) has no tool flags. When the model supports tool calling, the tools field works unchanged:

const res = await client.chat.completions.create({
  model: "<model-id>",
  messages: [{ role: "user", content: "What is the weather in Lisbon?" }],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        parameters: {
          type: "object",
          properties: { city: { type: "string" } },
          required: ["city"],
        },
      },
    },
  ],
});

If tool calls never come back for a model that you expect to support them, test the model with a small request and no client code. A model that does not support tools gives a plain text answer.

Using the official OpenAI SDKs

Point the SDK’s base URL at the inference base URL and pass your FirstByte API key. Nothing else changes. These are the snippets the console’s SDK quickstart card emits. Python

from openai import OpenAI

client = OpenAI(
    base_url=os.environ["INFERENCE_BASE_URL"],
    api_key=os.environ["CLUTCHCALL_CREDENTIALS"],
)

stream = client.chat.completions.create(
    model="<model-id>",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

Node

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: process.env.INFERENCE_BASE_URL,
  apiKey: process.env.CLUTCHCALL_CREDENTIALS,
});

const stream = await client.chat.completions.create({
  model: "<model-id>",
  messages: [{ role: "user", content: "Hello" }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Keep the API key on your backend. It is a long-lived, tenant-scoped credential; see Authentication for the pattern to use when a browser needs access.

Authentication — API keys and how they scope a caller
Modalities overview

Start here

Console

Guides

Reference

OpenAI-Compatible HTTP API

Base URL and authentication

Endpoint reference

Streaming over HTTP/3

Tool calling

Using the official OpenAI SDKs

​Base URL and authentication

​Endpoint reference

​Streaming over HTTP/3

​Tool calling

​Using the official OpenAI SDKs

​Related

Base URL and authentication

Endpoint reference

Streaming over HTTP/3

Tool calling

Using the official OpenAI SDKs

Related