Skip to main content
Configure multiple provider keys in an access key configuration routing rule. The gateway tries keys in order when one hits a rate limit, runs out of quota, or fails authentication.

​
How failover works

When a routing rule step lists multiple provider keys:
  1. The gateway tries the first key.
  2. On rate limit (429), auth failure, or certain upstream errors, it tries the next key.
  3. If all keys in the step fail, the gateway tries the next step in the rule (if any), or returns an error.
Failover is configured per provider in the access key configuration—not in your application code.

​
Via dashboard

  1. Add multiple provider keys for the same provider.
  2. Edit an access key configuration.
  3. Add or edit a routing rule for the provider.
  4. Attach multiple keys to the Bring your own API key step in the order you want them tried.

​
Via the AI Gateway API

curl -X POST https://api.ngrok.ai/access-key-configurations \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "OpenAI with failover",
    "router": {
      "rules": [
        {
          "provider": "openai",
          "steps": [
            {
              "type": "user",
              "keySelectionStrategy": "ordered",
              "keys": [
                { "id": "aigpk_primary" },
                { "id": "aigpk_backup" }
              ]
            }
          ]
        }
      ]
    }
  }'
See the Access Key Configurations API reference. Assign the configuration to your access key. Your app sends only the access key: 
client = OpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx",
)

​
Next steps