OpenAI Proxy

The MetrixLLM gateway provides a drop-in proxy for OpenAI-compatible endpoints. Use your existing OpenAI SDKs or raw HTTP requests by changing only the Base URL and API key.

Base URL

https://gateway.metrixllm.com

Authentication

All requests require a MetrixLLM API key. The gateway accepts both formats:

Authorization: Bearer sk-metrix-YOUR_API_KEY

x-api-key: sk-metrix-YOUR_API_KEY

Do not use your OpenAI API key. The gateway authenticates requests using MetrixLLM keys (sk-metrix-* prefix). The gateway then uses its own provider keys (or your BYOK keys) to call OpenAI.

Internal Authentication

For backend-to-gateway communication, the gateway supports internal authentication:

Header	Required	Description
`x-metrix-internal`	Yes	Shared secret matching `HEALTH_CHECK_SECRET` (timing-safe comparison)
`x-metrix-workspace-id`	Yes*	Workspace UUID (use `x-workspace-id` as fallback)

Internal auth skips the standard API key validation. Used by the MetrixLLM backend for playground, replay, and batch operations.

Endpoints

Method	Path	Description
`POST`	`/v1/chat/completions`	Generic chat completions (MetrixLLM format)
`POST`	`/openai/v1/chat/completions`	Chat completions (OpenAI-native format)
`POST`	`/openai/v1/embeddings`	Embeddings (OpenAI-native format)
`POST`	`/v1/embeddings`	Embeddings (MetrixLLM format)
`GET`	`/v1/models`	List available models

Request Headers

Every request supports these headers:

Header	Type	Required	Description
`Authorization`	`string`	Yes*	`Bearer sk-metrix-{key}` — MetrixLLM API key
`x-api-key`	`string`	Yes*	`sk-metrix-{key}` — alternative to `Authorization`
`Content-Type`	`string`	Yes	Must be `application/json`

Session Tracing Use these headers to group related requests into sessions and trace spans:

Header	Description	Stored In
`metrix-session-id`	Unique session identifier — groups related requests into a traceable session	`workspace_logs.session_id`
`metrix-parent-id`	Links a request to a parent span in the session trace	`workspace_logs.parent_id`
`metrix-session-name`	Human-readable name for the span (for dashboard display)	`workspace_logs.span_name`

Custom Properties Attach up to 10 custom metadata properties per request:

Header Pattern	Description
`metrix-property-{key}`	Custom metadata property. Used for rate limit targeting, analytics, and filtering.

Example: metrix-property-team: engineering, metrix-property-environment: production Worker-to-Gateway These headers are set by the Cloudflare Edge Worker when routing through the Durable Object balance system:

Header	Description
`x-balance`	Workspace credit balance from Durable Object
`x-internal-signature`	HMAC-SHA256 signature of `{workspace_id}:{balance}` verified with `DO_SYNC_SECRET`

IP Detection

Header	Description
`x-forwarded-for`	Client IP address (first IP in comma-separated list). Trusted only when peer is a private/local address.
`x-real-ip`	Fallback client IP header. Same trust rules as `x-forwarded-for`.
`cf-ipcountry`	Cloudflare-specific country code of the connecting client (e.g. `US`, `IN`). Used for request logging and analytics.

Chat Completions

Request Body

Send standard OpenAI Chat Completions requests. The gateway accepts all standard OpenAI fields plus MetrixLLM-specific extensions.

Field	Type	Required	Description
`model`	`string`	Yes	Model identifier (e.g. `gpt-4o`, `gpt-4o-mini`, `o3`)
`messages`	`array`	Yes	Array of message objects
`max_tokens`	`integer`	No	Maximum completion tokens
`temperature`	`number`	No	Sampling temperature (0-2)
`top_p`	`number`	No	Nucleus sampling parameter
`frequency_penalty`	`number`	No	Frequency penalty (-2 to 2)
`presence_penalty`	`number`	No	Presence penalty (-2 to 2)
`stop`	`string \| array`	No	Stop sequences
`stream`	`boolean`	No	Enable streaming (see Streaming)
`tools`	`array`	No	Function/tool definitions
`tool_choice`	`string \| object`	No	Tool selection strategy
`reasoning_effort`	`string`	No	Reasoning effort level: `low`, `medium`, `high` (for o-series models)
`mode`	`string`	No	Key selection: `"auto"` (default), `"byok"`, or `"native"`
`web_search`	`boolean \| string`	No	Enable web search: `true` (standard), `"native"` (provider built-in)
`web_search_max_uses`	`integer`	No	Max web search queries per request (native mode)
`web_search_options`	`object`	No	Native search options (context size, user location)
`web_search_level`	`string`	No	Search depth: `"low"`, `"medium"`, `"high"`
`web_search_lang`	`string`	No	Search language code
`web_search_country`	`string`	No	Search country code
`prompt_id`	`string`	No	Use a stored prompt template
`inputs`	`object`	No	Template variables for prompt placeholders
`route`	`string`	No	Routing override for the provider path

Examples

curl -X POST https://gateway.metrixllm.com/openai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-metrix-YOUR_API_KEY" \
  -H "metrix-session-id: session_abc123" \
  -H "metrix-session-name: Customer Onboarding" \
  -H "metrix-property-environment: production" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the capital of France?"}
    ],
    "max_tokens": 1024,
    "temperature": 0.7
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.metrixllm.com/openai",
    api_key="sk-metrix-YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"},
    ],
    max_tokens=1024,
    extra_headers={
        "metrix-session-id": "session_abc123",
        "metrix-session-name": "Customer Onboarding",
        "metrix-property-environment": "production",
    },
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.metrixllm.com/openai",
  apiKey: "sk-metrix-YOUR_API_KEY",
  defaultHeaders: {
    "metrix-session-id": "session_abc123",
    "metrix-session-name": "Customer Onboarding",
    "metrix-property-environment": "production",
  },
});

const response = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is the capital of France?" },
  ],
  max_tokens: 1024,
});

console.log(response.choices[0].message.content);

Response Body

{
  "id": "req_aK3x9mP2",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 8,
    "total_tokens": 33
  }
}

Response Headers

Every successful response includes these headers:

Header	Description
`Content-Type`	Always `application/json`
`X-Metrix-Provider`	Resolved upstream provider (e.g. `openai`, `anthropic`, `google`)
`X-Metrix-Model`	Actual model used for the request
`X-Metrix-Cache-Status`	`"HIT"` or `"miss"` (semantic cache)
`X-Metrix-Retry-Count`	Number of retry attempts (0 if none)
`X-Metrix-Retry-Status`	`"success"`, `"failed"`, or `"not_triggered"`
`X-Metrix-Fallback`	`"true"` if a fallback provider was used (only present when applicable)
`X-RateLimit-Limit`	Configured limit for the rate window
`X-RateLimit-Remaining`	Remaining quota in the current window
`X-RateLimit-Requests-Remaining`	Remaining requests in the current rate window
`X-RateLimit-Tokens-Remaining`	Remaining tokens in the current rate window

On cache HIT responses, only Content-Type and X-Metrix-Cache-Status: HIT are returned. On rate limited responses (429 Too Many Requests):

Header	Description
`Retry-After`	Seconds to wait before retrying
`X-RateLimit-{type}-Limit`	Maximum limit (e.g. `X-RateLimit-rpm-Limit`, `X-RateLimit-tpm-Limit`)
`X-RateLimit-{type}-Remaining`	Remaining count in the current window
`X-RateLimit-Reset`	Seconds until the rate window resets

Streaming

Set "stream": true in the request body to receive Server-Sent Events (SSE).

curl -X POST https://gateway.metrixllm.com/openai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-metrix-YOUR_API_KEY" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "Write a haiku about coding."}
    ],
    "stream": true
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.metrixllm.com/openai",
    api_key="sk-metrix-YOUR_API_KEY",
)

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a haiku about coding."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        print(delta.content, end="", flush=True)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.metrixllm.com/openai",
  apiKey: "sk-metrix-YOUR_API_KEY",
});

const stream = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Write a haiku about coding." }],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process.stdout.write(content);
}

Streaming is supported through the gateway proxy endpoints (/v1/chat/completions and /openai/v1/chat/completions). The gateway returns SSE events in the standard OpenAI format.

Web Search

Enable web search to give the model access to real-time information.

Value	Behavior
`true`	Performs a web search and injects results into the system prompt
`"native"`	Uses the provider’s built-in web search tool
`false` or absent	No web search

Additional parameters:

Field	Type	Default	Description
`web_search_max_uses`	`integer`	Provider default	Maximum number of search queries per request
`web_search_level`	`string`	`"medium"`	Search depth: `"low"` (1 query), `"medium"` (2 queries), `"high"` (3 queries)
`web_search_lang`	`string`	`"en"`	Search language code
`web_search_country`	`string`	`"us"`	Search country code

Native Web Search Example

curl -X POST https://gateway.metrixllm.com/openai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-metrix-YOUR_API_KEY" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "What are the latest developments in AI?"}
    ],
    "web_search": "native",
    "web_search_options": {
      "search_context_size": "medium",
      "user_location": {"type": "approximate", "city": "San Francisco"}
    }
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.metrixllm.com/openai",
    api_key="sk-metrix-YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What are the latest developments in AI?"}],
    extra_body={
        "web_search": "native",
        "web_search_options": {
            "search_context_size": "medium",
            "user_location": {"type": "approximate", "city": "San Francisco"},
        },
    },
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.metrixllm.com/openai",
  apiKey: "sk-metrix-YOUR_API_KEY",
});

const response = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "What are the latest developments in AI?" }],
  web_search: "native",
  web_search_options: {
    search_context_size: "medium",
    user_location: { type: "approximate", city: "San Francisco" },
  },
});

console.log(response.choices[0].message.content);

Prompt Management

Use prompt_id to reference a stored prompt template. The gateway fetches the template, renders {{variable}} placeholders with values from inputs, and sends the rendered messages to the provider.

{
  "model": "gpt-4o",
  "prompt_id": "prompt_abc123",
  "inputs": {
    "user_name": "Alice",
    "topic": "quantum computing"
  }
}

The prompt template must exist in your workspace. The inputs object replaces {{variable}} placeholders in each message’s content field.

Error Responses

Errors return a JSON body with an error field. The HTTP status code indicates the error type.

Status	Meaning
`400`	Invalid request (missing model, malformed body)
`401`	Invalid or missing API key
`402`	Insufficient credits
`403`	Workspace disabled, IP not allowed, or model not enabled
`404`	Model not recognized
`429`	Rate limit exceeded
`446`	Guardrail block (request blocked by content policy)
`500`	Internal gateway error
`502`	Upstream provider error
`503`	Provider circuit breaker open

Error Response Format

{
  "error": "Rate limit exceeded for requests.",
  "limit": 100,
  "windowMs": 60000
}

Retry and Fallback

The gateway automatically retries failed requests and falls back to alternative providers:

Retry: Retries on 429, 500, 502, 503, 504 status codes with exponential backoff
Fallback: If the primary provider fails, the gateway tries the next configured provider in the chain
Circuit Breaker: Providers with repeated failures are temporarily skipped (circuit opens for 60 seconds)
Cerebras Fallback: If all configured providers fail on system errors, the gateway falls back to Cerebras (if enabled)

These features are transparent to your application — you receive a single successful response.

Mode Override

Control provider key selection with the mode field:

Value	Behavior
`"auto"`	Default. Uses your BYOK key if configured, otherwise uses MetrixLLM native key.
`"byok"`	Forces use of your own API key (Bring Your Own Key). Fails if no key is configured for the provider.
`"native"`	Forces use of MetrixLLM’s managed key. Skips BYOK keys entirely.

{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Hello!"}],
  "mode": "native"
}

Overview

Connect

Build

Observe

Optimize

Manage

Gateway API Reference

Base URL

Authentication

Internal Authentication

Endpoints

Request Headers

Chat Completions

Request Body

Examples

Response Body

Response Headers

Streaming

Web Search

Native Web Search Example

Prompt Management

Error Responses

Error Response Format

Retry and Fallback

Mode Override

​Base URL

​Authentication

​Internal Authentication

​Endpoints

​Request Headers

​Chat Completions

​Request Body

​Examples

​Response Body

​Response Headers

​Streaming

​Web Search

​Native Web Search Example

​Prompt Management

​Error Responses

​Error Response Format

​Retry and Fallback

​Mode Override

Base URL

Authentication

Internal Authentication

Endpoints

Request Headers

Chat Completions

Request Body

Examples

Response Body

Response Headers

Streaming

Web Search

Native Web Search Example

Prompt Management

Error Responses

Error Response Format

Retry and Fallback

Mode Override