Quickstart
Send your first request in 5 minutes
Drop-in Endpoints
Just change your baseURL — no rewrites
Dashboard Guide
Explore the MetrixLLM UI
What is MetrixLLM?
MetrixLLM is an AI gateway that sits between your application and your LLM providers. Every request your app makes passes through the gateway, where MetrixLLM applies your routing rules, checks the semantic cache, evaluates guardrails, and logs full telemetry — all in real time with near-zero added latency. You do not need to rewrite any of your application code. MetrixLLM is fully compatible with the OpenAI, Anthropic, and Google Gemini SDKs. You change one line.Core pillars
Intelligent Routing
Define fallback chains. If your primary provider fails, MetrixLLM transparently retries the next one. Your app sees a single clean response.
Full Observability
Every request is logged: provider, model, tokens, cost, latency, prompt text, and response — filterable in real time from the dashboard.
Semantic Caching
Identical or semantically similar prompts are served from cache. No code changes required — enable it once in workspace settings.
Guardrails
Block or log requests matching keyword lists, regex patterns, or length constraints — before the upstream provider is ever called.
Cost Control
Set hard budget caps and rate limits per workspace or API key. Never get a surprise bill.
Prompt Management
Store, version, and serve prompts from the dashboard. Reference them by ID in requests — no deployments to update a prompt.
How the gateway processes a request
1
Authentication
The gateway validates your
mtx_ API key against your workspace and checks IP allowlists.2
Rate Limiting
Per-workspace and per-key rate limits (RPM/TPM) and budget caps are evaluated. Requests over the limit are rejected immediately.
3
Guardrails (Input)
Your configured input guardrails are evaluated against the prompt. Blocked requests never reach the upstream provider.
4
Cache Lookup
If semantic caching is enabled, the gateway checks for a matching cached response and returns it instantly if found.
5
Routing
The gateway selects the upstream provider and model based on your routing rules and fallback chain.
6
Upstream Call
The request is forwarded to the provider. On failure, the next fallback in your chain is tried automatically.
7
Guardrails (Output)
Response guardrails are evaluated before the response reaches your app.
8
Logging
Full telemetry (tokens, cost, latency, payloads, session data) is logged asynchronously — zero impact on response time.
Next steps
Quickstart →
Make your first request in 5 minutes
Supported Providers →
OpenAI, Anthropic, Google, xAI, Groq, Together AI