> ## Documentation Index
> Fetch the complete documentation index at: https://docs.metrixllm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Routing & Fallbacks

> Ensure high availability by automatically falling back to alternative providers when your primary provider goes down.

Provider outages, latency spikes, and rate limits are inevitable. MetrixLLM's routing engine lets your application survive these issues seamlessly without any changes to your code.

## How it works

When you send a request to the gateway, MetrixLLM evaluates your configured routing rules for that specific endpoint (e.g. `/v1/chat/completions`).

Instead of routing directly to a single model, MetrixLLM tries providers sequentially based on your **fallback chain**.

## Routing modes

Every request can specify a `mode` parameter to control how the gateway authenticates with providers:

| Mode               | Description                                                     | Balance deduction                     | Use case                                   |
| ------------------ | --------------------------------------------------------------- | ------------------------------------- | ------------------------------------------ |
| `"auto"` (default) | Prefers your BYOK key if configured, falls back to platform key | BYOK: none. Native: yes               | Most users — automatic cost optimization   |
| `"byok"`           | Forces your own API key for every provider                      | None — your key, your cost            | Cost-sensitive workloads, compliance       |
| `"native"`         | Forces MetrixLLM's platform key                                 | Yes — deducted from workspace credits | Quick testing, no provider accounts needed |

### How auto mode works

1. If you have a BYOK key configured for the requested provider, the gateway uses it (no charge).
2. If no BYOK key is configured, the gateway uses the platform key (deducted from workspace credits).
3. If your workspace has insufficient credits **and** you have a BYOK key, the gateway automatically uses your BYOK key to avoid a payment error.

### Custom providers

Requests routed to [Custom Providers](/docs/custom-providers) (Ollama, vLLM, LM Studio, etc.) are **always BYOK** regardless of the `mode` parameter. These requests never deduct workspace credits since they use your own infrastructure.

```json theme={null}
{
  "model": "llama-3.3-70b-versatile",
  "mode": "byok",
  "messages": [{"role": "user", "content": "Hello!"}]
}
```

## Configuring a fallback chain

<Steps>
  <Step title="Open Routing Rules">
    In your workspace, go to **Routing Rules** and click **New Rule**.
  </Step>

  <Step title="Add primary provider">
    Select the primary provider and model (e.g. `openai`, `gpt-4o`). This is the first provider MetrixLLM will always try.
  </Step>

  <Step title="Add fallback providers">
    Click **Add Fallback** and select your secondary provider (e.g. `anthropic`, `claude-opus-4-5`). You can add as many fallbacks as you want.
  </Step>

  <Step title="Configure retries">
    Set the **Max Retries** and select which HTTP status codes should trigger a retry (e.g. `429 Too Many Requests`, `500 Internal Server Error`).
  </Step>
</Steps>

## Retry Configuration

MetrixLLM supports automatic retries with configurable parameters:

| Parameter           | Default                     | Description                                   |
| :------------------ | :-------------------------- | :-------------------------------------------- |
| `enabled`           | `false`                     | Enable/disable automatic retries              |
| `max_retries`       | `2`                         | Maximum number of retry attempts per provider |
| `backoff_ms`        | `500`                       | Milliseconds to wait between retries          |
| `retry_on_statuses` | `[429, 500, 502, 503, 504]` | HTTP status codes that trigger a retry        |

Retries happen within the same provider before falling back to the next provider in the chain.

## What your application sees

Your application does not need to handle the fallback logic. If `openai` is down and returns a `502 Bad Gateway`, MetrixLLM automatically catches the error, retries the exact same prompt with `anthropic`, and returns the successful Anthropic response back to your app.

Your code receives a normal HTTP 200 response, exactly as if the first provider had succeeded.

### Response headers

To help you understand what happened under the hood, the gateway injects metadata headers into every response:

| Header                  | Description                                                                                               |
| :---------------------- | :-------------------------------------------------------------------------------------------------------- |
| `X-Metrix-Provider`     | The provider that ultimately served the request                                                           |
| `X-Metrix-Model`        | The exact model used                                                                                      |
| `X-Metrix-Fallback`     | `true` if a fallback provider was used                                                                    |
| `X-Metrix-Retry-Count`  | The total number of retries attempted across the chain                                                    |
| `X-Metrix-Retry-Status` | The outcome of retry attempts: `"success"` if a retry succeeded, `"not_triggered"` if no retries occurred |
| `X-Metrix-Cache-Status` | `"HIT"` or `"MISS"` indicating cache behavior                                                             |

## Customizing behavior per request

If you want to override the dashboard routing rules for a specific request, you can use the `prompt_id` mechanism (see [Prompt Management](/docs/prompt-management)) or define custom endpoint targets in the Custom Provider settings.

## Programmatic routing example

You can inspect response headers to understand routing decisions:

```javascript theme={null}
const response = await fetch('https://gateway.metrixllm.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer sk-metrix-...',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Hello!' }],
  }),
});

// Check which provider served the request
const provider = response.headers.get('X-Metrix-Provider');
const model = response.headers.get('X-Metrix-Model');
const retryCount = response.headers.get('X-Metrix-Retry-Count');
const retryStatus = response.headers.get('X-Metrix-Retry-Status');
const wasFallback = response.headers.get('X-Metrix-Fallback');

console.log(`Served by: ${provider}/${model}`);
console.log(`Retries: ${retryCount}, Status: ${retryStatus}`);
console.log(`Fallback used: ${wasFallback}`);
```