---
title: "Providers and models"
---

# Providers and models

Connect model providers to the CLI, keep their keys out of files, choose models per role, and set fallbacks, budgets and reasoning effort.

Models are named `provider/model`, for example `anthropic/claude-sonnet-5-5`, `openai/gpt-5`, `ollama/qwen3-coder` or `openrouter/anthropic/claude-sonnet-4.5`. Everything after the first `/` is the model id. Choose one with `--model` (`-m`), the `model` setting, or `/model` in a session. `neuralinverse models [provider]` lists the models available from your configured providers.

## Set up a provider

Start `neuralinverse` with no usable provider and it walks you through setup instead of exiting. Run `neuralinverse setup [provider]` to do it again, or `/provider add` in a session. (`-p` with no provider exits with an error that names the variables to set.)

1. **Provider:** Anthropic, OpenAI, Google Gemini, OpenRouter, AWS Bedrock, Vertex AI, Azure OpenAI, Ollama, or any other OpenAI-compatible server.
2. **Credentials,** checked live with a model listing before anything is saved; no tokens are spent.
   - **API key:** typed into a hidden field; a rejected key can be retried. If the provider's usual variable (for example `ANTHROPIC_API_KEY`) is already set, setup offers to reference it with `apiKeyEnv`.
   - **OpenRouter:** sign in with the browser (OAuth with PKCE, a one-time listener on `localhost` receives the code) or paste a key.
   - **Ollama:** the local server (`OLLAMA_HOST` or `localhost:11434`) is detected and its installed models listed.
   - **Bedrock, Vertex, Azure:** existing AWS credentials and profiles (`~/.aws`), Google application-default credentials and the gcloud project, and `AZURE_OPENAI_ENDPOINT` are detected; you pick the profile and region, the project and region, or the resource and deployment. Vertex has no listing API, so it is checked with a one-token request to the model you choose.
3. **Default model** from the provider's live list, recommended models first.
4. **Permission mode** and, where available, the **shell sandbox**.

Everything is written to `~/.neuralinverse/settings.json` at the end. Stopping earlier writes nothing.

### Where keys are kept

Keys never go into `settings.json`. Setup stores them in the operating system's keychain and references them with `apiKeyCommand`:

| OS | Stored with | Settings reference |
|---|---|---|
| macOS | `security add-generic-password` (login keychain, service `neuralinverse`) | `security find-generic-password -s neuralinverse -a <provider> -w` |
| Linux | `secret-tool store` (GNOME Keyring or KWallet via the Secret Service; key passed on standard input) | `secret-tool lookup service neuralinverse account <provider>` |
| Windows | PowerShell SecretManagement `Set-Secret`, when a vault is registered | `pwsh -NoProfile -NonInteractive -Command "Get-Secret -Name 'neuralinverse-<provider>' -AsPlainText"` |

When no keychain works (no Secret Service session, a locked keychain, no vault), the key goes into `~/.neuralinverse/credentials.json` (mode 0600) and the provider gets `"apiKeyFile": "~/.neuralinverse/credentials.json"`. `cmdkey` is not used on Windows because a stored password cannot be read back.

### Manage providers in a session

| Command | Effect |
|---|---|
| `/provider` | Providers in use (configured and detected) and how each gets its key (never the key itself) |
| `/provider add [type]` | The setup steps for one provider; usable at once |
| `/provider switch [id]` | List that provider's models live and switch; optionally save as the default |
| `/provider test [id]` | Run the live check again (default: the current provider) |
| `/provider remove <id>` | Remove it from your user settings and delete its stored key |

## Providers that need no configuration

These are available as soon as their credentials are present:

| Provider id | Kind | Credentials |
|---|---|---|
| `anthropic` | anthropic | `ANTHROPIC_API_KEY` |
| `openai` | openai | `OPENAI_API_KEY` |
| `gemini` | gemini | `GEMINI_API_KEY` or `GOOGLE_API_KEY` |
| `bedrock` | bedrock | `AWS_REGION` plus AWS credentials (environment, profile, container or instance role), or `AWS_BEARER_TOKEN_BEDROCK` |
| `vertex` | vertex-anthropic | `ANTHROPIC_VERTEX_PROJECT_ID` or `GOOGLE_CLOUD_PROJECT`, `CLOUD_ML_REGION`, Google application-default credentials |
| `openrouter`, `deepseek`, `groq`, `mistral`, `xai` | openai | `OPENROUTER_API_KEY`, `DEEPSEEK_API_KEY`, `GROQ_API_KEY`, `MISTRAL_API_KEY`, `XAI_API_KEY` |
| `ollama` | openai | None; uses `OLLAMA_HOST` or `http://localhost:11434` |

Without a `model` setting, the first available of anthropic, openai, gemini, bedrock, vertex, openrouter, deepseek, mistral, xai and groq is used with a default model.

## Configure a provider

Add entries under `providers`; the key becomes the provider id.

```jsonc
{
  "model": "corp/gpt-5",
  "providers": {
    // Azure OpenAI: `model` is the deployment name.
    "corp": {
      "kind": "openai",
      "baseUrl": "https://my-resource.openai.azure.com",
      "azureApiVersion": "2025-04-01-preview",
      "apiKeyEnv": "AZURE_OPENAI_API_KEY"
    },
    // A self-hosted vLLM server with a key from a secrets manager.
    "gpu": {
      "kind": "openai",
      "baseUrl": "https://vllm.internal.example.com/v1",
      "apiKeyCommand": "op read op://eng/vllm/token",
      "models": { "qwen3-coder-480b": { "contextWindow": 262144, "maxOutputTokens": 32768 } }
    },
    "bedrock-eu": { "kind": "bedrock", "region": "eu-central-1", "awsProfile": "prod" }
  }
}
```

| Field | Purpose |
|---|---|
| `kind` | `anthropic`, `openai` (Chat Completions and compatible servers), `openai-responses`, `gemini`, `bedrock`, `vertex-anthropic` |
| `baseUrl` | API endpoint override |
| `apiKey` | The key itself. Avoid this in any file that might be shared |
| `apiKeyEnv` | Environment variable holding the key |
| `apiKeyCommand` | Command that prints the key; run once per process, 10-second timeout |
| `apiKeyFile` | File holding the key, or the credentials file. With `apiKeyCommand` too, the file is the fallback when the command fails |
| `headers` | Extra HTTP headers; `${ENV}` is expanded |
| `azureApiVersion` | Switches an `openai` provider to Azure conventions |
| `region`, `awsProfile`, `gcpProject` | Cloud provider settings |
| `timeoutMs`, `maxRetries` | Request inactivity timeout (default 10 minutes) and retry count (default 4) |
| `models` | Per-model overrides: `contextWindow`, `maxOutputTokens`, `pricing` (USD per million tokens), `supportsImages`, `supportsReasoning` |

To route all model traffic through an internal gateway, define a custom `openai`, `anthropic` or `bedrock` provider with its `baseUrl`, and lock it with [managed settings](https://neuralinverse.com/docs/cli/enterprise/).

## Models per role

Every model call has a role, and each role can use its own model with `models.<role>`. Unset roles fall back to a neighbour, and finally to the main model. `model` and `smallModel` remain aliases for `models.main` and `models.fast`.

| Role | Used for |
|---|---|
| `main` | Main agent turns |
| `plan` | Read-only (plan mode) turns and `/deep-plan` |
| `fast` | Cheap background work |
| `subagent` | Delegated helper agents |
| `summary` | Summaries |
| `vision` | Prompts with images |

```jsonc
{ "models": { "main": "anthropic/claude-sonnet-5-5", "fast": "ollama/qwen3-coder", "plan": "anthropic/claude-opus-5-5" } }
```

## Fallback models

`fallback` replays a failed request on the next model of a chain, when the failure happened before any output reached you.

| Key | Meaning |
|---|---|
| `fallback.models` | The chain of `provider/model` refs |
| `fallback.on` | Failure kinds that trigger a fallback: `rate_limit`, `overloaded`, `server_error`, `timeout`, `auth`, `payment`. Default: the first four |
| `fallback.cooldownMs` | How long a failed model is skipped before it is tried again (default 60000) |

Once output has streamed nothing is replayed, so you never see duplicate text or tool calls. Aborts, context overflows and bad requests never trigger a fallback.

## Budgets

`budget` stops the agent gracefully at a spending or time limit. Limits are checked before every model call; the response in flight and the tool calls it requested still complete.

| Key | Meaning |
|---|---|
| `budget.perPrompt.usd`, `.tokens`, `.minutes`, `.toolCalls` | Limits for everything one prompt causes |
| `budget.perSession.usd`, `.tokens`, `.minutes` | Limits for the whole session |
| `budget.warnAt` | Fraction of a limit at which a warning is shown once (default 0.8) |

`--max-cost <usd>` and `--max-time <minutes>` set the per-session cost and time limits on the command line; with `--print`, reaching one exits with code `3`. The cost limit is skipped, with a notice, when pricing for the model is unknown.

## Reasoning, caching and cost

- `reasoning` (`off`, `low`, `medium`, `high`; also `--reasoning` and `/effort`) maps to each provider's extended-thinking or reasoning-effort controls where supported.
- `harness.effortRouting` (off by default) picks the effort from the request when no `reasoning` level is set: questions low, focused changes medium, multi-file or debugging requests high.
- Prompt caching is on by default for providers that support it; `"promptCaching": false` turns it off.
- `maxOutputTokens` (default 16384, capped by the model) and `temperature` apply where supported.
- Costs shown by `/usage` and the turn footer are estimates from built-in list prices, overridable per model. Your provider's bill is authoritative. Local models show no cost.

## Networking

`HTTPS_PROXY`, `HTTP_PROXY` and `NO_PROXY` are honoured for all provider traffic. Add corporate root certificates with `NODE_EXTRA_CA_CERTS`. Failed requests are retried with exponential backoff, honouring `Retry-After`, only when nothing has streamed yet.

## Models from your organisation

An organisation's signed policy can list the models people may use (`models.allowed`), add an organisation model gateway as the provider `keystone` (called with your Govern access token; its models show as "managed by <org>"), and turn off personal providers (`models.personalModels: false`). A model outside the list is refused in `/model`, `--model`, routing and the fallback chain with "not allowed by your organization's policy". The policy format is in [Enterprise deployment](https://neuralinverse.com/docs/cli/enterprise/#signed-organisation-policy).

This depends on the organisation policy reaching the CLI after sign-in, which the current Govern server does not yet do. See [Sign in to Govern](https://neuralinverse.com/docs/cli/sign-in/#known-gaps). To pin models today, lock `model` and `providers` in managed settings.