The model gateway is where Govern enforces model access. Every model request from the Neural Inverse apps, your applications, agents and routed coding tools goes through it, and it decides each one against your organisation's policy before anything reaches a provider. It runs inside the Govern server, next to the database, so budgets and rate limits hold across any number of replicas and provider keys never leave the process that holds the vault.
Pages in this part
| Task | Page |
|---|---|
| Connect a provider, approve models and assign them | Model access |
| Gateway keys, keyless workload identity, coding tool tokens, authorizers | Gateway keys and workload identity |
| Spend and token budgets, rate limits | Budgets and rate limits |
| Routes, fallbacks, retries, circuit breaker and response cache | Routing and caching |
| DLP and guardrails on prompts and answers | Guardrails and DLP |
| Request logs, provider health and attribution | Request logs and observability |
Where to configure it
The console has two services for this:
- Models: Providers and keys, Model catalog, Routing, Budgets, Assistant, and a Gateway group with Gateway endpoints, Connect a coding tool, Routes and strategies, Response cache, Guardrails, Rate limits, Request log settings and Provider health. Request logs, LLM traces and Model usage are under Monitor.
- AI Gateway: Gateways, Endpoints and domains, Gateway keys, Workload identity, Authorizers, Protection, Usage plans, Policies, Routing and reliability, Regions, Self-hosted nodes, governance pages and observability pages.
The in-server gateway described in these docs is working. The AI Gateway service also manages dedicated gateways with their own host names served by a separate data plane; the deployment of that data plane is recorded as partial in the source, and self-hosted gateway nodes are partial too. These docs describe the in-server gateway.
How a request is decided
Every request to /api/v1/models/gateway/v1/* goes through the same steps, in order:
- Authenticate the caller (
models:read) and parse the body. - Resolve the caller's teams, project, classification, policy, catalog, assignments, settings and routes in one read-only transaction.
- Find the candidates: the named model and its fallback chain, or the targets of the named route. Each candidate must be usable by the caller for the request's use and classification.
- Apply DLP to the prompt at the level the classification requires: redact or refuse.
- Check rate limits, then the response cache.
- Dispatch to each candidate in turn: skip it while its circuit is open, check its rate limits, reserve the worst case on every budget, call the provider, retry transient failures.
- Relay the answer in the caller's wire format, settle actual usage, audit the attempt, store the answer in the cache when allowed.
Steps 1 to 4 fail closed. Caching, retries and routing only save or repeat a provider call; none of them can let through a request that steps 1 to 4 refuse.
Endpoints
The base URL is your API URL followed by /api/v1/models/gateway. Anthropic clients use the base URL as it is; OpenAI clients add /v1.
| Method and path | Format |
|---|---|
POST /api/v1/models/gateway/v1/chat/completions | OpenAI chat completions |
POST /api/v1/models/gateway/v1/completions | OpenAI legacy completions |
POST /api/v1/models/gateway/v1/embeddings | OpenAI embeddings |
POST /api/v1/models/gateway/v1/responses | OpenAI Responses API |
GET /api/v1/models/gateway/v1/models | Models and routes the caller may use |
POST /api/v1/models/gateway/v1/messages | Anthropic Messages |
POST /api/v1/models/gateway/v1/messages/count_tokens | Anthropic token counting |
POST /api/v1/models/gateway/v1/prompts/{prompt}/chat/completions | Prompt library template, OpenAI format |
POST /api/v1/models/gateway/v1/prompts/{prompt}/messages | Prompt library template, Anthropic format |
POST /api/v1/models/gateway/v1/rerank | Rerank (openai_compatible and ollama connections) |
POST /api/v1/models/gateway/v1/images/generations | OpenAI image generation |
POST /api/v1/models/gateway/v1/audio/transcriptions | OpenAI transcription |
POST /api/v1/models/gateway/v1/audio/speech | OpenAI text to speech |
GET /api/v1/models/gateway/endpoints lists the same table as your deployment serves it, with full URLs. Request bodies are limited to 32 MiB. The Responses API accepts only function tools: tools the provider runs itself (web search, file search, code execution, remote MCP servers, computer use) are refused, because their traffic would never pass through DLP and the guardrails.
Provider-native paths are available for clients pinned to a provider's own API: POST /api/v1/models/gateway/anthropic/v1/messages and POST /api/v1/models/gateway/openai/v1/chat/completions (also /v1/completions, /v1/embeddings, /v1/responses). They run through the same checks and are off until an administrator turns them on; see Gateway keys and workload identity.
Example
curl -sS "$KEYSTONE_URL/api/v1/models/gateway/v1/chat/completions" \
-H "Authorization: Bearer $GATEWAY_KEY" \
-H "X-Keystone-Project: $PROJECT_ID" \
-H "X-Keystone-Use: chat" \
-H 'Content-Type: application/json' \
-d '{"model": "claude-sonnet-5-5", "messages": [{"role": "user", "content": "Summarise this change."}]}'The model field is a model key from your catalog or a route key.
Who can call it
The gateway accepts every Govern credential: a signed-in person's token, OAuth access tokens, API keys and agents, as long as the caller holds models:read. It adds its own:
| Credential | For |
|---|---|
| Gateway keys | Applications. A key that can call the gateway and nothing else, with its own scope, models, budgets and limits. |
| Workload identity | Workloads that exchange a token they already have (Kubernetes, CI, a Govern agent, a client certificate) for a short-lived gateway key. |
| Gateway tokens | Coding tools used by a person: a short-lived token bound to the person's session or enrolled device. |
| Authorizers | Per-route decisions by JWT, a custom HTTP or CEL authorizer, or none, for traffic without a Govern credential. |
See Gateway keys and workload identity.
Request headers
| Header | Effect |
|---|---|
X-Keystone-Project | Decide the request in this project: its classification, policies, assignments and budgets |
X-Keystone-Use | chat, agent, completion or embedding; the rerank, image and audio endpoints use rerank, image and audio |
X-Keystone-Classification | Raise the request's classification; it can never lower it |
X-Keystone-Metadata | A JSON object of attribution, at most 16 keys and 8192 bytes |
X-Keystone-Tags | Up to 10 tags |
X-Keystone-Retries | Lower the retries of this request |
X-Keystone-Cache-Control | no-cache, no-store |
X-Keystone-Client | The calling app and version, for the audit trail (advisory) |
X-Keystone-Context-Request-Id | The context retrieval this request used, linked in the evidence chain |
Metadata and tags are DLP-scanned like the prompt and never sent to the provider.
Response headers
| Header | Value |
|---|---|
X-Keystone-Request-Id | The request id (later attempts are <id>.<n>) |
X-Keystone-Model | The catalog model that answered |
X-Keystone-Route | The route, when the request named one |
X-Keystone-Attempts | Upstream attempts made (0 for a cache hit) |
X-Keystone-Cache | hit, semantic_hit, miss, bypass or off |
X-Keystone-Budget-Warning | A budget passed its warning threshold or a soft limit |
X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset | When a rate limit rule applied |
Retry-After | With circuit_open and rate_limited |
Server-Timing | gw;dur=<ms>: the gateway's own time before the response headers |
Refusal codes
Refusals use each wire format's own error shape and carry X-Keystone-Request-Id.
| Code | Meaning |
|---|---|
model_not_found, model_not_allowed, use_not_supported | The model or route is unknown, not assigned to the caller, or not for this use |
classification_not_allowed, region_not_allowed | The model may not serve this classification or region |
project_forbidden, project_required | The caller is not in the project, or the organisation requires one |
dlp_blocked, dlp_unavailable | DLP refused the prompt or answer, or could not run |
guardrail_blocked | A guardrail refused the prompt or answer |
budget_exceeded, plan_limit | A hard budget or the plan's limit would be crossed |
rate_limited | A rate limit rule refused the request |
circuit_open | Every candidate's circuit is open |
no_eligible_model, provider_disabled, provider_unavailable, provider_auth_failed, key_unavailable | No usable model or provider |
gateway_suspended | The organisation's gateway was suspended by the AI security kill switch |
invalid_request, service_unavailable | Malformed request, or a dependency could not be reached |
A provider's own error body never reaches the caller, because it can quote the prompt. The caller gets a generic refusal and the request id; operators get the body with DLP redaction applied.
For the Neural Inverse apps
The Editor and the Agents app fetch a signed view of what the person may use from GET /api/v1/models/resolved?project=<project id> and verify it with the public keys at /.well-known/neuralinverse-models-keys.json. The manifest only shapes what the app offers; the gateway decides every request again. The manifest signing key is set with NI_AUTH_MODELS_SIGNING_KEY_FILE (or _PEM) and NI_AUTH_MODELS_SIGNING_KEY_ID; without it the endpoint answers 503 models.signing_unavailable and no unsigned manifest is served.
Audit
Every attempt is audited as models.gateway.request with the decision, model, classification, tokens, cost, route and attribution, but no prompt content. Refusals are audited with their code. Configuration changes are audited in their own transaction.