---
title: "Budgets and rate limits"
---

# Budgets and rate limits

Cap model spend and tokens per organisation, team, project, person or key in Neural Inverse Govern, and limit request and token rates over sliding windows.

Budgets cap how much is spent or how many tokens are used in a day or month. Rate limits cap how fast requests or tokens flow. Both are enforced by the model gateway on every request, before anything is sent to a provider, and both hold across any number of server replicas.

## Budgets

Open **Models > Budgets**, or use `POST /api/v1/models/budgets` (`models:write`).

| Field | Values |
|---|---|
| Scope | `org`, `team`, `project` or `user`. A `user` budget with no user named applies to every person separately ("each developer 50 dollars a month"). |
| Metric | `spend` (micro-dollars) or `tokens` |
| Period | `day` or `month`, UTC |
| Limit | The cap |
| Warning threshold | Optional, in percent |
| Enforcement | `hard` stops at the limit; `soft` only warns |

Gateway keys and authorizer usage plans can carry their own budgets on top; see [Gateway keys and workload identity](https://neuralinverse.com/docs/govern/gateway-keys/).

### How enforcement works

1. **Reserve.** Before a request is sent, the gateway reserves its worst case on every applicable budget in one database transaction, with the counters locked in a fixed order. A request that would cross a hard limit is refused with `429 budget_exceeded`. Concurrent requests can never reserve more than a hard limit.
2. **Worst case.** Input tokens are estimated at one token per three bytes of prompt text plus fixed overheads (1,600 tokens per image). Output tokens are the request's `max_tokens`, capped by the model's `max_output_tokens`; when the client sends none, the gateway sets it to the model's maximum. Requests for several completions reserve once per completion.
3. **Settle.** After the response, the reservation is replaced with the usage the provider reported. Reported usage is never capped: an overrun is charged, logged and audited. If the provider reports no usage, the estimate is charged.
4. **Unfinished requests.** A reservation not settled in time is settled at its reserved amount by a sweep (`NI_AUTH_MODELS_RESERVATION_HOLD`, default `30m`). Charging the worst case is the fail-closed choice.

Crossing a warning threshold or a soft limit adds `X-Keystone-Budget-Warning` to the response, and the first crossing in a period emits a `models.budget.warning` event.

Spend on a person's own provider key is not counted against spend budgets, because it is not the organisation's money; tokens are counted. Cache hits reserve a token and a micro-dollar, so a used-up hard budget refuses them too.

`GET /api/v1/models/budgets` returns each budget with its live usage (`models:view`). Settled reservations are kept for `NI_AUTH_MODELS_RESERVATION_RETENTION` (default 90 days); budget counters, usage events and the audit log keep the totals.

Model Access budgets are the only AI budgets Govern enforces. The older cost explorer AI budgets were imported into them and their routes answer `410`.

## Rate limits

Open **Models > Rate limits**, or use `POST /api/v1/models/rate-limits` (`models:admin`).

```bash
curl -sS -X POST "$KEYSTONE_URL/api/v1/models/rate-limits" \
  -H "Authorization: Bearer $ADMIN_TOKEN" -H 'Content-Type: application/json' \
  -d '{"name": "Interactive users", "subject_kind": "user", "metric": "tokens",
       "window_seconds": 60, "limit": 200000}'
```

| Field | Rule |
|---|---|
| `name` | 1 to 100 characters |
| `subject_kind` | `user`, `api_key`, `project`, `team`, `org` or `tag` |
| `subject_id` | Left out for `org`; required for `tag`; otherwise optional. Without it the rule applies to every user, key, project or team separately. |
| `model` | Optional model key or route key |
| `metric` | `requests` or `tokens` |
| `window_seconds` | 1 to 86400 |
| `limit` | 1 to 1000000000000 |
| `enabled` | Default `true` |

An organisation has at most 200 rules.

### How enforcement works

- A request is checked against every enabled rule that applies; all must admit it.
- A rule on a model key counts every request sent to that model however it was named: directly, as a fallback, as a route target or from its cached answer. Naming a route or another model never avoids it. A model whose rule refuses is skipped like a model whose circuit is open; the request is refused with `429 rate_limited` and `Retry-After` when no candidate can serve it.
- `requests` cost 1 per request that reaches dispatch. `tokens` cost the request's estimate before dispatch and are corrected to actual usage afterwards.
- Counters use a sliding window and are checked and charged in one PostgreSQL transaction with rows locked, so two requests can never both take the last of a limit on any number of replicas.
- If the counter store cannot be read or locked in time, the request is refused with `503 service_unavailable`. Rate limiting fails closed.
- Responses carry `X-RateLimit-Limit`, `X-RateLimit-Remaining` and `X-RateLimit-Reset` for the rule with the least room left.

### Tag rules are attribution, not enforcement

The client decides whether to send a tag in `X-Keystone-Tags`, so a request that leaves it out is not counted by the tag's rule. Use tag rules to shape the traffic of cooperating clients, such as a batch job. For a limit that must hold, use a `user`, `api_key`, `project`, `team` or `org` rule.

## Routes

| Route | Permission |
|---|---|
| `GET /api/v1/models/budgets` | `models:view` |
| `POST /api/v1/models/budgets`, `PATCH`, `DELETE /api/v1/models/budgets/{id}` | `models:write` |
| `GET /api/v1/models/rate-limits` | `models:view` |
| `POST /api/v1/models/rate-limits`, `PATCH`, `DELETE /api/v1/models/rate-limits/{id}` | `models:admin` |

Changing a rule's window or model resets its counters; disabling it stops counting at once. Every change is audited (`models.rate_limit.create`, `.update`, `.delete`).

## Plan limits on the hosted service

On the hosted service, plan limits also apply (for example the daily gateway tokens). A self-hosted installation has no plan limits.