> ## Documentation Index
> Fetch the complete documentation index at: https://ngrok.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Multi-Provider Failover

> Fail over between models and providers in AI Gateway requests.

You can configure fail over by listing multiple models in a request. The gateway tries each model in order until one succeeds. Combine this with an [access key configuration](/ai-gateway/guides/access-key-configurations) that allows the providers you need.

Your app sends an [access key](/ai-gateway/concepts/access-keys); the AI Gateway routes to each provider with the credentials from your configuration.

## Basic example

```python theme={null}
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx",
)

response = client.chat.completions.create(
    model="gpt-4o",
    extra_body={
        "models": ["anthropic:claude-sonnet-4-6"],
    },
    messages=[{"role": "user", "content": "Hello!"}],
)
```

Or in raw JSON:

```json theme={null}
{
  "model": "gpt-4o",
  "models": ["anthropic:claude-sonnet-4-6"],
  "messages": [{"role": "user", "content": "Hello"}]
}
```

## How it works

1. Request arrives with your access key
2. Gateway tries `gpt-4o` (OpenAI)
3. On failure, tries `anthropic:claude-sonnet-4-6`
4. Returns the first successful response

Both providers must be allowed by the access key's configuration, with routing rules that supply credentials (ngrok.ai inference or provider keys).

## Configuration example

Allow OpenAI and Anthropic on one key:

```json theme={null}
{
  "name": "Multi-provider production",
  "access": {
    "providers": { "allow": ["openai", "anthropic"] }
  },
  "router": {
    "rules": [
      { "provider": "openai", "steps": [{ "type": "ngrok" }] },
      { "provider": "anthropic", "steps": [{ "type": "ngrok" }] }
    ]
  }
}
```

Use provider keys instead of `ngrok` steps when billing should go to your provider accounts.

## Next steps

* [Choose a model](/ai-gateway/guides/model-selection-strategies): Model IDs and provider prefixes
* [Configure fallback models](/ai-gateway/guides/configure-fallback-models): Fallback ordering and access rules
* [Multi-key failover](/ai-gateway/guides/key-selection-failover): Fail over between provider keys
* [Access Key Configurations](/ai-gateway/guides/access-key-configurations): Scope and routing