> ## Documentation Index
> Fetch the complete documentation index at: https://docs.metrixllm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI Proxy

> Route OpenAI-compatible requests through MetrixLLM for observability, rate limiting, and provider fallback.

The MetrixLLM gateway provides a drop-in proxy for OpenAI-compatible endpoints. Use your existing OpenAI SDKs or raw HTTP requests by changing only the Base URL and API key.

## Base URL

```text theme={null}
https://gateway.metrixllm.com
```

## Authentication

All requests require a MetrixLLM API key. The gateway accepts both formats:

```text theme={null}
Authorization: Bearer sk-metrix-YOUR_API_KEY
```

```text theme={null}
x-api-key: sk-metrix-YOUR_API_KEY
```

<Warning>
  Do **not** use your OpenAI API key. The gateway authenticates requests using MetrixLLM keys (`sk-metrix-*` prefix). The gateway then uses its own provider keys (or your BYOK keys) to call OpenAI.
</Warning>

### Internal Authentication

For backend-to-gateway communication, the gateway supports internal authentication:

| Header                  | Required | Description                                                           |
| ----------------------- | -------- | --------------------------------------------------------------------- |
| `x-metrix-internal`     | Yes      | Shared secret matching `HEALTH_CHECK_SECRET` (timing-safe comparison) |
| `x-metrix-workspace-id` | Yes\*    | Workspace UUID (use `x-workspace-id` as fallback)                     |

Internal auth skips the standard API key validation. Used by the MetrixLLM backend for playground, replay, and batch operations.

## Endpoints

| Method | Path                          | Description                                 |
| ------ | ----------------------------- | ------------------------------------------- |
| `POST` | `/v1/chat/completions`        | Generic chat completions (MetrixLLM format) |
| `POST` | `/openai/v1/chat/completions` | Chat completions (OpenAI-native format)     |
| `POST` | `/openai/v1/embeddings`       | Embeddings (OpenAI-native format)           |
| `POST` | `/v1/embeddings`              | Embeddings (MetrixLLM format)               |
| `GET`  | `/v1/models`                  | List available models                       |

***

## Request Headers

Every request supports these headers:

| Header          | Type     | Required | Description                                        |
| --------------- | -------- | -------- | -------------------------------------------------- |
| `Authorization` | `string` | Yes\*    | `Bearer sk-metrix-{key}` — MetrixLLM API key       |
| `x-api-key`     | `string` | Yes\*    | `sk-metrix-{key}` — alternative to `Authorization` |
| `Content-Type`  | `string` | Yes      | Must be `application/json`                         |

**Session Tracing**

Use these headers to group related requests into sessions and trace spans:

| Header                | Description                                                                  | Stored In                   |
| --------------------- | ---------------------------------------------------------------------------- | --------------------------- |
| `metrix-session-id`   | Unique session identifier — groups related requests into a traceable session | `workspace_logs.session_id` |
| `metrix-parent-id`    | Links a request to a parent span in the session trace                        | `workspace_logs.parent_id`  |
| `metrix-session-name` | Human-readable name for the span (for dashboard display)                     | `workspace_logs.span_name`  |

**Custom Properties**

Attach up to 10 custom metadata properties per request:

| Header Pattern          | Description                                                                        |
| ----------------------- | ---------------------------------------------------------------------------------- |
| `metrix-property-{key}` | Custom metadata property. Used for rate limit targeting, analytics, and filtering. |

Example: `metrix-property-team: engineering`, `metrix-property-environment: production`

**Worker-to-Gateway**

These headers are set by the Cloudflare Edge Worker when routing through the Durable Object balance system:

| Header                 | Description                                                                        |
| ---------------------- | ---------------------------------------------------------------------------------- |
| `x-balance`            | Workspace credit balance from Durable Object                                       |
| `x-internal-signature` | HMAC-SHA256 signature of `{workspace_id}:{balance}` verified with `DO_SYNC_SECRET` |

**IP Detection**

| Header            | Description                                                                                                          |
| ----------------- | -------------------------------------------------------------------------------------------------------------------- |
| `x-forwarded-for` | Client IP address (first IP in comma-separated list). Trusted only when peer is a private/local address.             |
| `x-real-ip`       | Fallback client IP header. Same trust rules as `x-forwarded-for`.                                                    |
| `cf-ipcountry`    | Cloudflare-specific country code of the connecting client (e.g. `US`, `IN`). Used for request logging and analytics. |

***

## Chat Completions

### Request Body

Send standard OpenAI Chat Completions requests. The gateway accepts all standard OpenAI fields plus MetrixLLM-specific extensions.

| Field                 | Type                | Required | Description                                                           |
| --------------------- | ------------------- | -------- | --------------------------------------------------------------------- |
| `model`               | `string`            | Yes      | Model identifier (e.g. `gpt-4o`, `gpt-4o-mini`, `o3`)                 |
| `messages`            | `array`             | Yes      | Array of message objects                                              |
| `max_tokens`          | `integer`           | No       | Maximum completion tokens                                             |
| `temperature`         | `number`            | No       | Sampling temperature (0-2)                                            |
| `top_p`               | `number`            | No       | Nucleus sampling parameter                                            |
| `frequency_penalty`   | `number`            | No       | Frequency penalty (-2 to 2)                                           |
| `presence_penalty`    | `number`            | No       | Presence penalty (-2 to 2)                                            |
| `stop`                | `string \| array`   | No       | Stop sequences                                                        |
| `stream`              | `boolean`           | No       | Enable streaming (see [Streaming](#streaming))                        |
| `tools`               | `array`             | No       | Function/tool definitions                                             |
| `tool_choice`         | `string \| object`  | No       | Tool selection strategy                                               |
| `reasoning_effort`    | `string`            | No       | Reasoning effort level: `low`, `medium`, `high` (for o-series models) |
| `mode`                | `string`            | No       | Key selection: `"auto"` (default), `"byok"`, or `"native"`            |
| `web_search`          | `boolean \| string` | No       | Enable web search: `true` (standard), `"native"` (provider built-in)  |
| `web_search_max_uses` | `integer`           | No       | Max web search queries per request (native mode)                      |
| `web_search_options`  | `object`            | No       | Native search options (context size, user location)                   |
| `web_search_level`    | `string`            | No       | Search depth: `"low"`, `"medium"`, `"high"`                           |
| `web_search_lang`     | `string`            | No       | Search language code                                                  |
| `web_search_country`  | `string`            | No       | Search country code                                                   |
| `prompt_id`           | `string`            | No       | Use a stored prompt template                                          |
| `inputs`              | `object`            | No       | Template variables for prompt placeholders                            |
| `route`               | `string`            | No       | Routing override for the provider path                                |

### Examples

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST https://gateway.metrixllm.com/openai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer sk-metrix-YOUR_API_KEY" \
    -H "metrix-session-id: session_abc123" \
    -H "metrix-session-name: Customer Onboarding" \
    -H "metrix-property-environment: production" \
    -d '{
      "model": "gpt-4o",
      "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"}
      ],
      "max_tokens": 1024,
      "temperature": 0.7
    }'
  ```

  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://gateway.metrixllm.com/openai",
      api_key="sk-metrix-YOUR_API_KEY",
  )

  response = client.chat.completions.create(
      model="gpt-4o",
      messages=[
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "What is the capital of France?"},
      ],
      max_tokens=1024,
      extra_headers={
          "metrix-session-id": "session_abc123",
          "metrix-session-name": "Customer Onboarding",
          "metrix-property-environment": "production",
      },
  )

  print(response.choices[0].message.content)
  ```

  ```javascript Node.js theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://gateway.metrixllm.com/openai",
    apiKey: "sk-metrix-YOUR_API_KEY",
    defaultHeaders: {
      "metrix-session-id": "session_abc123",
      "metrix-session-name": "Customer Onboarding",
      "metrix-property-environment": "production",
    },
  });

  const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: "What is the capital of France?" },
    ],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

### Response Body

```json theme={null}
{
  "id": "req_aK3x9mP2",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 8,
    "total_tokens": 33
  }
}
```

### Response Headers

Every successful response includes these headers:

| Header                           | Description                                                             |
| -------------------------------- | ----------------------------------------------------------------------- |
| `Content-Type`                   | Always `application/json`                                               |
| `X-Metrix-Provider`              | Resolved upstream provider (e.g. `openai`, `anthropic`, `google`)       |
| `X-Metrix-Model`                 | Actual model used for the request                                       |
| `X-Metrix-Cache-Status`          | `"HIT"` or `"miss"` (semantic cache)                                    |
| `X-Metrix-Retry-Count`           | Number of retry attempts (0 if none)                                    |
| `X-Metrix-Retry-Status`          | `"success"`, `"failed"`, or `"not_triggered"`                           |
| `X-Metrix-Fallback`              | `"true"` if a fallback provider was used (only present when applicable) |
| `X-RateLimit-Limit`              | Configured limit for the rate window                                    |
| `X-RateLimit-Remaining`          | Remaining quota in the current window                                   |
| `X-RateLimit-Requests-Remaining` | Remaining requests in the current rate window                           |
| `X-RateLimit-Tokens-Remaining`   | Remaining tokens in the current rate window                             |

On cache HIT responses, only `Content-Type` and `X-Metrix-Cache-Status: HIT` are returned.

On rate limited responses (`429 Too Many Requests`):

| Header                         | Description                                                           |
| ------------------------------ | --------------------------------------------------------------------- |
| `Retry-After`                  | Seconds to wait before retrying                                       |
| `X-RateLimit-{type}-Limit`     | Maximum limit (e.g. `X-RateLimit-rpm-Limit`, `X-RateLimit-tpm-Limit`) |
| `X-RateLimit-{type}-Remaining` | Remaining count in the current window                                 |
| `X-RateLimit-Reset`            | Seconds until the rate window resets                                  |

***

## Streaming

Set `"stream": true` in the request body to receive Server-Sent Events (SSE).

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST https://gateway.metrixllm.com/openai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer sk-metrix-YOUR_API_KEY" \
    -d '{
      "model": "gpt-4o",
      "messages": [
        {"role": "user", "content": "Write a haiku about coding."}
      ],
      "stream": true
    }'
  ```

  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://gateway.metrixllm.com/openai",
      api_key="sk-metrix-YOUR_API_KEY",
  )

  stream = client.chat.completions.create(
      model="gpt-4o",
      messages=[{"role": "user", "content": "Write a haiku about coding."}],
      stream=True,
  )

  for chunk in stream:
      delta = chunk.choices[0].delta
      if delta.content:
          print(delta.content, end="", flush=True)
  ```

  ```javascript Node.js theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://gateway.metrixllm.com/openai",
    apiKey: "sk-metrix-YOUR_API_KEY",
  });

  const stream = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Write a haiku about coding." }],
    stream: true,
  });

  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content;
    if (content) process.stdout.write(content);
  }
  ```
</CodeGroup>

<Note>
  Streaming is supported through the gateway proxy endpoints (`/v1/chat/completions` and `/openai/v1/chat/completions`). The gateway returns SSE events in the standard OpenAI format.
</Note>

***

## Web Search

Enable web search to give the model access to real-time information.

| Value             | Behavior                                                         |
| ----------------- | ---------------------------------------------------------------- |
| `true`            | Performs a web search and injects results into the system prompt |
| `"native"`        | Uses the provider's built-in web search tool                     |
| `false` or absent | No web search                                                    |

Additional parameters:

| Field                 | Type      | Default          | Description                                                                   |
| --------------------- | --------- | ---------------- | ----------------------------------------------------------------------------- |
| `web_search_max_uses` | `integer` | Provider default | Maximum number of search queries per request                                  |
| `web_search_level`    | `string`  | `"medium"`       | Search depth: `"low"` (1 query), `"medium"` (2 queries), `"high"` (3 queries) |
| `web_search_lang`     | `string`  | `"en"`           | Search language code                                                          |
| `web_search_country`  | `string`  | `"us"`           | Search country code                                                           |

### Native Web Search Example

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST https://gateway.metrixllm.com/openai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer sk-metrix-YOUR_API_KEY" \
    -d '{
      "model": "gpt-4o",
      "messages": [
        {"role": "user", "content": "What are the latest developments in AI?"}
      ],
      "web_search": "native",
      "web_search_options": {
        "search_context_size": "medium",
        "user_location": {"type": "approximate", "city": "San Francisco"}
      }
    }'
  ```

  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://gateway.metrixllm.com/openai",
      api_key="sk-metrix-YOUR_API_KEY",
  )

  response = client.chat.completions.create(
      model="gpt-4o",
      messages=[{"role": "user", "content": "What are the latest developments in AI?"}],
      extra_body={
          "web_search": "native",
          "web_search_options": {
              "search_context_size": "medium",
              "user_location": {"type": "approximate", "city": "San Francisco"},
          },
      },
  )

  print(response.choices[0].message.content)
  ```

  ```javascript Node.js theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://gateway.metrixllm.com/openai",
    apiKey: "sk-metrix-YOUR_API_KEY",
  });

  const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What are the latest developments in AI?" }],
    web_search: "native",
    web_search_options: {
      search_context_size: "medium",
      user_location: { type: "approximate", city: "San Francisco" },
    },
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

***

## Prompt Management

Use `prompt_id` to reference a stored prompt template. The gateway fetches the template, renders `{{variable}}` placeholders with values from `inputs`, and sends the rendered messages to the provider.

```json theme={null}
{
  "model": "gpt-4o",
  "prompt_id": "prompt_abc123",
  "inputs": {
    "user_name": "Alice",
    "topic": "quantum computing"
  }
}
```

The prompt template must exist in your workspace. The `inputs` object replaces `{{variable}}` placeholders in each message's `content` field.

***

## Error Responses

Errors return a JSON body with an `error` field. The HTTP status code indicates the error type.

| Status | Meaning                                                  |
| ------ | -------------------------------------------------------- |
| `400`  | Invalid request (missing model, malformed body)          |
| `401`  | Invalid or missing API key                               |
| `402`  | Insufficient credits                                     |
| `403`  | Workspace disabled, IP not allowed, or model not enabled |
| `404`  | Model not recognized                                     |
| `429`  | Rate limit exceeded                                      |
| `446`  | Guardrail block (request blocked by content policy)      |
| `500`  | Internal gateway error                                   |
| `502`  | Upstream provider error                                  |
| `503`  | Provider circuit breaker open                            |

### Error Response Format

```json theme={null}
{
  "error": "Rate limit exceeded for requests.",
  "limit": 100,
  "windowMs": 60000
}
```

***

## Retry and Fallback

The gateway automatically retries failed requests and falls back to alternative providers:

* **Retry**: Retries on `429`, `500`, `502`, `503`, `504` status codes with exponential backoff
* **Fallback**: If the primary provider fails, the gateway tries the next configured provider in the chain
* **Circuit Breaker**: Providers with repeated failures are temporarily skipped (circuit opens for 60 seconds)
* **Cerebras Fallback**: If all configured providers fail on system errors, the gateway falls back to Cerebras (if enabled)

These features are transparent to your application -- you receive a single successful response.

***

## Mode Override

Control provider key selection with the `mode` field:

| Value      | Behavior                                                                                             |
| ---------- | ---------------------------------------------------------------------------------------------------- |
| `"auto"`   | Default. Uses your BYOK key if configured, otherwise uses MetrixLLM native key.                      |
| `"byok"`   | Forces use of your own API key (Bring Your Own Key). Fails if no key is configured for the provider. |
| `"native"` | Forces use of MetrixLLM's managed key. Skips BYOK keys entirely.                                     |

```json theme={null}
{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Hello!"}],
  "mode": "native"
}
```