Neural Inverse Govern / AI Gateway
Every model request decided against your policy before it reaches a provider.
One gateway inside Govern for people, applications, agents and coding tools, compatible with the OpenAI and Anthropic APIs. Provider keys never leave it.
- Endpoints
- OpenAI- and Anthropic-compatible
- Providers
- Hosted APIs, cloud platforms and self-hosted models
- Models
- Deny by default
- Budgets
- Reserved before the request is sent
- Runs
- Inside the Govern server
Overview
The AI Gateway is where Govern enforces model access. It authenticates the caller, resolves their teams, project and data classification, applies DLP and your guardrails, checks rate limits and budgets, and only then calls a provider with a key the caller never sees. Every attempt is audited with who asked, which model answered, the tokens and the cost, without the prompt.
What it does
What the AI Gateway does.
Model access, credentials, content checks, spend and reliability, decided in one place.
ProvidersAnthropic, OpenAI, Google, Azure OpenAI, Amazon Bedrock, Vertex AI, and self-hosted servers that speak the OpenAI API or Ollama. Provider keys are held in Govern's vault, are write-only and are never returned.
Approved modelsNothing is usable by default. A model reaches a person only when its connection is enabled, it is in your catalog, it is allowed at a scope that applies to them, and it is allowed for the data's classification. A deny at any scope wins.
Classification and regionsEach request takes the strictest classification of its project, team, department and organisation; a client can raise it, never lower it. Restricted work defaults to self-hosted or zero-retention models, and models can be limited to regions.
Your own catalogYou enter each model and its prices; Govern does not discover models from providers. Prices stay under your control, and the catalog works air-gapped.
Gateway keysKeys that can call the gateway and nothing else, scoped to the organisation, a team, a project or a person, with their own models, budgets, rate limits, network allowlist, expiry, chargeback metadata, rotation with overlap and per-key analytics.
Keyless workload identityWorkloads exchange a credential they already have, such as a Kubernetes or CI token, an agent's credentials or a client certificate, for a key that lives at most an hour. No static secret is deployed.
Coding tool tokensCoding tools send requests as the person using them, with a token of at most one hour bound to their session or enrolled device. It works only on the gateway's model endpoints.
AuthorizersPer-route decisions for traffic without a Govern credential: JWT, your own HTTP endpoint or a CEL expression, with optional client certificates and signed requests. Admitting callers without a credential requires a rate limit and a hard spend budget.
Data loss preventionEvery text part of the prompt is scanned. Findings are redacted at public and internal, and refused at confidential, restricted or in a regulated project. Content that cannot be read as text is refused at those levels, not passed unscanned.
GuardrailsYour rules on top of model access: input deny and allow patterns, an injection heuristic, mandated system text, output DLP, output deny patterns and JSON schema checks. Streamed answers are checked before they are sent. Guardrails only add checks.
DetectorsOptional local detectors for sensitive data with reversible tokenisation, scored prompt injection, runaway agents and destructive tool calls, which can hold a call for approval. They start in shadow mode and are heuristics: review findings before you enforce them.
BudgetsSpend or token budgets per organisation, team, project, person or key, by day or month, hard or soft. The worst case is reserved before a request is sent, so concurrent requests cannot exceed a hard limit.
Rate limitsRequest and token limits per person, key, project, team or organisation, optionally per model, over sliding windows. A limit on a model counts every request it serves, however it was named.
Routing and reliabilityRetries with backoff, a circuit breaker per model, fallback chains, and routes with fallback, weighted, latency, cost, round-robin, least-busy and consistent-hash strategies and conditions. A fallback never reaches a model the caller could not use directly.
Response cacheOff by default. Exact or semantic, scoped to a person, project or organisation, sealed under your organisation's key, and served only to callers who could have made the same request.
Compatible endpointsOpenAI chat, completions, embeddings and Responses, Anthropic Messages and token counting, rerank, images and audio, so existing SDKs and tools work by changing the base URL. Tools a provider would run itself are refused, because their traffic would bypass DLP.
Request logsOff by default. Metadata only, or full bodies after DLP sealed under your organisation's key, never above a classification you set and never for a regulated project. Reading bodies is itself audited.
How it works
How a model request is decided.
Every request takes the same steps, in order. The first four fail closed; caching, retries and routing can never let through what they refuse.
- AuthenticateThe caller is identified from a Govern credential, a gateway key, a workload exchange or an authorizer.
- ResolveTeams, project, classification, policy, catalog, assignments and routes, read in one transaction.
- Check contentDLP at the level the classification requires, then your guardrails: redact or refuse.
- LimitRate limits, then the response cache.
- Reserve and callThe worst case is reserved on every budget, then the provider is called with a key from the vault.
- RecordActual usage is settled and the attempt is audited, without prompt content.
Who uses it
Built for every role that answers for AI.
Platform and AI enablement teamsConnect providers, curate the catalog and its prices, assign models and build routes with fallbacks.
Security teamsSet classification and region rules, guardrails and detectors, and read request logs and refusals.
Finance and engineering managersSet budgets per team, project or person, and read spend per key, model and project.
Application developersCall models with a gateway key or workload identity, using existing OpenAI and Anthropic SDKs.
Enforced and recorded
What the gateway refuses, and what it records.
Model access, content checks, budgets and rate limits are enforced on the server for every request that reaches the gateway.
Enforced
Model accessAssignment, classification and region are checked for the requested model, every fallback and every route target.
Fails closedIf the DLP scan cannot run, the request is refused. If the rate limit counters cannot be read in time, it is refused.
Hard budgetsReserved in one database transaction with counters locked, so they hold across any number of server replicas.
Provider errorsA provider's own error body never reaches the caller, because it can quote the prompt. The caller gets a generic refusal and a request id.
Kill switchThe AI security kill switch suspends an organisation's gateway, and every request is refused until it is lifted.
Recorded
Every attemptAudited with the decision, model, classification, tokens, cost, route and attribution, and guardrail results. Refusals are audited with their code.
Attribution headersRequest metadata and tags are DLP-scanned and recorded, never sent to the provider. The calling app's own name is recorded as advisory; the OAuth client of the token is recorded as verified.
Usage and analyticsUsage events, per-key analytics, model usage and provider health.
Soft budgets only warn, and rate limits on tags count only clients that send the tag. Use hard budgets, and limits on people, keys, projects, teams or the organisation, for anything that must hold. The gateway governs the traffic that reaches it; without an egress lock on developer machines, a person can reach a provider with another key.
Inside the Govern serverThe gateway runs in the server process, next to the database, so provider keys never leave the process that holds the vault.
Self-hosted modelsOllama and OpenAI-compatible servers on networks the operator opens count as self-hosted, for work that must not leave your network.
Air-gappedWith local models only. The catalog and prices are yours, and the response cache and full request logs need the deployment's key management service.
Hosted by usWe run it for you. Plan limits such as daily gateway tokens also apply; a self-hosted installation has none.
Get started
Set it up with the documentation.
Talk to our team.
Neural Inverse is available by request. We work with your engineering, security and procurement teams from the first conversation.