Skip to main content
The Routing Rules page lets you define how the gateway handles requests for a specific route — including which providers to try, in what order, and how to retry on failure.

​
Creating a Routing Rule

  1. Click New Rule.
  2. Enter a Route Name — this should match the route path in your requests (e.g. /v1/chat/completions).
  3. Add one or more providers/models to the chain. The first entry is the primary; any subsequent entries are fallbacks.
  4. Configure retry settings if desired.
  5. Set rate limits or budget caps for this specific route.
  6. Click Save.

​
Provider Chain

The provider chain is an ordered list. If the first provider fails (5xx error, timeout, or rate limit), the gateway automatically tries the next entry in the chain. Example chain:
  1. openai → gpt-4o (primary)
  2. anthropic → claude-opus-4-5 (fallback)
  3. groq → llama-3.3-70b-versatile (last resort)

​
Retry Configuration

You can configure automatic retries for transient failures (e.g. 429 Too Many Requests or 503 Service Unavailable):
  • Max Retries — Maximum number of retry attempts
  • Retry on Status Codes — Which HTTP error codes should trigger a retry

​
Rate Limits per Route

In addition to workspace-level rate limits, you can set route-specific limits to throttle a specific endpoint independently. See Rate Limits & Budgets for more detail.