Custom Providers

​

What is a Custom Provider?

• Are per-workspace — each workspace can have its own custom endpoints
• Support the OpenAI chat completions format (/v1/chat/completions)
• Can be local (no API key needed) or remote (with authentication)
• Appear alongside built-in providers in routing configuration
• Support model registration so you can track costs and capabilities

​

Supported Servers

​

Setup

​

1. Add a Custom Provider

• Provider Name — A human-readable label (e.g., “My Local Llama”)
• Base URL — The server’s base URL (e.g., http://localhost:11434/v1)
• API Key — Optional. Leave empty for local servers without authentication

​

2. Register Models

• Model ID — The model name the server expects (e.g., llama-3.3-70b)
• Display Name — A human-friendly name (e.g., “Llama 3.3 70B (Local)”)
• Context Window — Maximum context length (e.g., 128K)

​

3. Test the Connection

​

Using Custom Providers

​

Via the Gateway API

curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Authorization: Bearer sk-metrix-your-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.3-70b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

​

Via the Dashboard Playground

1. Go to Playground
2. Select your custom model from the model dropdown
3. Send a test message

​

Via the Drop-in Endpoint

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="sk-metrix-your-key"
)

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Hello!"}]
)

​

How Routing Works

1. The gateway looks up the model name in admin_models (built-in providers)
2. If not found, it checks your workspace’s custom providers
3. If a match is found, the request is forwarded to your custom endpoint
4. The response is parsed using OpenAI format and returned to the client

​

Caching and Invalidation

​

Limitations

• Custom providers only support the chat completions format (/v1/chat/completions)
• Embedding endpoints are not yet supported for custom providers
• Web search injection is not available for custom provider models
• Custom provider models are workspace-scoped — other workspaces cannot see them

​

API Reference

​

List Custom Providers

GET /api/workspaces/:workspaceId/custom-providers

​

Create Custom Provider

POST /api/workspaces/:workspaceId/custom-providers
Body: { "name": "My Llama", "baseUrl": "http://localhost:11434/v1", "apiKey": "optional" }

​

Update Custom Provider

PUT /api/workspaces/:workspaceId/custom-providers/:providerId
Body: { "name": "Updated Name", "baseUrl": "http://new-url:8000/v1" }

​

Delete Custom Provider

DELETE /api/workspaces/:workspaceId/custom-providers/:providerId

​

Test Connection

POST /api/workspaces/:workspaceId/custom-providers/:providerId/test

​

Add Model

POST /api/workspaces/:workspaceId/custom-providers/:providerId/models
Body: { "modelName": "llama-3.3-70b", "displayName": "Llama 3.3 70B", "contextWindow": "128K" }

​

Delete Model

DELETE /api/workspaces/:workspaceId/custom-providers/:providerId/models/:modelId