Prompt Management

Creating a prompt

In the dashboard, navigate to Prompts and click New Prompt.

Write your system prompt, including variables wrapped in {{ }} brackets. For example:

You are a helpful customer support agent for {{ company_name }}. Answer the following question: {{ user_query }}

When you save, the prompt is assigned a unique UUID (the prompt_id), and version v1 is created.

Using a prompt in your code

To use the prompt, you send a standard request to the gateway, but instead of passing the raw messages array, you pass the prompt_id and the inputs object.

The gateway resolves the prompt template, injects the variables, formats the messages, and forwards it to the provider.

{
  "model": "gpt-4o",
  "prompt_id": "123e4567-e89b-12d3-a456-426614174000",
  "inputs": {
    "company_name": "MetrixLLM",
    "user_query": "How do I reset my password?"
  }
}

{
  "model": "claude-opus-4-5",
  "max_tokens": 1024,
  "prompt_id": "123e4567-e89b-12d3-a456-426614174000",
  "inputs": {
    "company_name": "MetrixLLM",
    "user_query": "How do I reset my password?"
  }
}

Versioning

Every time you edit a prompt in the dashboard, a new version is created (v2, v3, etc).

The gateway always resolves requests to the latest version of the prompt. If you push a bad prompt, you can instantly revert live traffic by going to the prompt’s version history and clicking Set as Latest on an older version.

For performance, the gateway caches resolved prompt templates in memory. Changes made in the dashboard may take up to 5 minutes (300s) to fully propagate to all live traffic.

Overview

Connect

Build

Observe

Optimize

Manage

Gateway API Reference

Creating a prompt

Using a prompt in your code

Versioning

​Creating a prompt

​Using a prompt in your code

​Versioning

Creating a prompt

Using a prompt in your code

Versioning