> ## Documentation Index
> Fetch the complete documentation index at: https://docs.metrixllm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Semantic Caching

> Reduce latency by 90% and save costs by serving identical or semantically similar prompts from the edge.

MetrixLLM's Semantic Cache stores responses from your LLM providers. When a new request arrives, the gateway generates an embedding of the prompt and checks the cache.

If an exact match or a highly similar prompt is found, MetrixLLM returns the cached response instantly without calling the upstream provider.

## Enabling the cache

Caching requires **zero code changes** in your application.

<Steps>
  <Step title="Go to Settings">
    In your workspace, navigate to **Settings**.
  </Step>

  <Step title="Enable Caching">
    Toggle **Semantic Cache** to Enabled.
  </Step>
</Steps>

Once enabled, the gateway automatically intercepts and caches responses for all requests passing through that workspace.

## How matching works

MetrixLLM supports two types of cache matching:

### Exact match

The fastest path. If the incoming prompt is character-for-character identical to a cached prompt (including system message, model, and temperature), the cached response is returned immediately with zero embedding overhead.

### Semantic similarity

For prompts that aren't exact matches but are semantically equivalent, the gateway uses vector embeddings to find similar cached responses. This handles cases like:

* Rephrased questions ("What is the capital of France?" vs "Which city is the capital of France?")
* Minor wording differences
* Different instruction phrasings with identical intent

## Configuration options

| Setting                  | Default | Description                                                                                                    |
| :----------------------- | :------ | :------------------------------------------------------------------------------------------------------------- |
| **Similarity Threshold** | `0.92`  | Minimum cosine similarity score (0-1) required for a semantic cache hit. Higher values require closer matches. |
| **TTL (Time To Live)**   | `3600`  | Seconds before a cached entry expires. Set to `0` for no expiration.                                           |
| **Max Cache Size**       | `10000` | Maximum number of entries stored per workspace. Oldest entries are evicted when full.                          |

### Tuning the similarity threshold

| Threshold     | Behavior                                        | Use case                                    |
| :------------ | :---------------------------------------------- | :------------------------------------------ |
| `0.95 - 1.0`  | Very strict — only near-identical prompts match | Factual queries where exact wording matters |
| `0.88 - 0.94` | Balanced — catches rephrased questions          | General-purpose chat and Q\&A               |
| `0.80 - 0.87` | Loose — broader semantic matching               | Creative or exploratory prompts             |

<Warning>
  Setting the threshold too low may return semantically different responses. Start with the default (`0.92`) and adjust based on your cache hit rate and response quality.
</Warning>

## Identifying cache hits

When a response is served from the cache, your application receives it just like a normal response, but it will be much faster.

You can definitively identify a cache hit by checking the response headers:

| Header                  | Value           |
| :---------------------- | :-------------- |
| `X-Metrix-Cache-Status` | `HIT` or `MISS` |

## Viewing cache analytics

The dashboard highlights exactly how much money and time caching saves you.

Navigate to **Cost Analytics** in your dashboard. At the top of the page, the **Cache Savings** card shows:

* The number of requests served from cache
* The total tokens saved
* The estimated USD saved based on the model's pricing
* The average latency reduction