> ## Documentation Index
> Fetch the complete documentation index at: https://ngrok.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Gateway Overview

> Route requests to AI providers with automatic failover, load balancing, and observability.

The ngrok AI Gateway ([ngrok.ai](https://ngrok.ai)) enables your applications to call both hosted AI providers and local models through a single endpoint. Control who has access, decide what they can call, and choose whether requests use credits, your provider keys, or a model you run yourself.

## Why use the AI Gateway?

<CardGroup cols={2}>
  <Card title="Start without provider accounts" icon="key">
    Use built-in OpenAI and Anthropic with an access key and credits—no OpenAI or Anthropic signup required.
  </Card>

  <Card title="Route to Self-Hosted Models" icon="server">
    Route to local models like Ollama or vLLM alongside cloud providers with [a model you run yourself](/ai-gateway/guides/use-a-model-you-run-yourself).
  </Card>

  <Card title="Automatic Failover" icon="rotate">
    If one provider or key fails, ngrok.ai automatically tries the next model or key.
  </Card>

  <Card title="Compatible With Popular SDKs" icon="plug">
    Works with official and third-party SDKs. Point the base URL at `gateway.ngrok.ai` and use your access key.
  </Card>
</CardGroup>

## Quick example

Sign up at [app.ngrok.ai](https://app.ngrok.ai) and point your SDK at the gateway with your [access key](/ai-gateway/concepts/access-keys):

<CodeGroup>
  ```python Python (OpenAI) highlight={4-5} theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://gateway.ngrok.ai/v1",
      api_key="ng-xxxxx-g1-xxxxx"  # Your access key
  )

  response = client.chat.completions.create(
      model="gpt-4o",
      messages=[{"role": "user", "content": "Hello!"}]
  )
  ```

  ```python Python (Anthropic) highlight={4-5} theme={null}
  import anthropic

  client = anthropic.Anthropic(
      base_url="https://gateway.ngrok.ai",
      api_key="ng-xxxxx-g1-xxxxx"  # Your access key
  )

  message = client.messages.create(
      model="claude-opus-4-6",
      max_tokens=1024,
      messages=[{"role": "user", "content": "Hello!"}]
  )
  ```

  ```typescript TypeScript (OpenAI) highlight={4-5} theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://gateway.ngrok.ai/v1",
    apiKey: "ng-xxxxx-g1-xxxxx"  // Your access key
  });

  const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }]
  });
  ```

  ```typescript TypeScript (Anthropic) highlight={4-5} theme={null}
  import Anthropic from "@anthropic-ai/sdk";

  const client = new Anthropic({
    baseURL: "https://gateway.ngrok.ai",
    apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
  });

  const message = await client.messages.create({
    model: "claude-opus-4-6",
    max_tokens: 1024,
    messages: [{ role: "user", content: "Hello!" }],
  });
  ```

  ```bash cURL (OpenAI) highlight={3} theme={null}
  curl https://gateway.ngrok.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer ng-xxxxx-g1-xxxxx" \
    -d '{
      "model": "gpt-4o",
      "messages": [{"role": "user", "content": "Hello!"}]
    }'
  ```

  ```bash cURL (Anthropic) highlight={3} theme={null}
  curl https://gateway.ngrok.ai/v1/messages \
    -H "Content-Type: application/json" \
    -H "x-api-key: ng-xxxxx-g1-xxxxx" \
    -H "anthropic-version: 2023-06-01" \
    -d '{
      "model": "claude-opus-4-5",
      "max_tokens": 1024,
      "messages": [{"role": "user", "content": "Hello!"}]
    }'
  ```
</CodeGroup>

You don't need to sign up for OpenAI or bring your own OpenAI key to make this request. Your ngrok.ai credits will cover any provider costs.

<Note>
  Every request consumes [credits](/ai-gateway/concepts/credits). No subscription plan required to get started. See [Credits](/ai-gateway/concepts/credits) for more details and when you might need a ngrok subscription for features like team members or internal endpoints.
</Note>

On each request, the AI Gateway:

1. Receives your request with your access key at `gateway.ngrok.ai`
2. Validates the key and loads your account configuration
3. Selects which model and provider to use
4. Forwards the request using ngrok.ai inference or your provider keys
5. Retries with the next key or model on failure
6. Returns the response

## What can you do?

| Use Case                          | Description                                                                                                              |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| **Use without provider accounts** | Get started with OpenAI and Anthropic using an access key and [credits](/ai-gateway/concepts/credits)                    |
| **Per-key routing**               | [Access key configurations](/ai-gateway/guides/access-key-configurations) scope providers, models, and credentials       |
| **Multi-model failover**          | List multiple models in a request; the gateway tries each in order                                                       |
| **Multi-key failover**            | Attach multiple provider keys in a configuration routing rule                                                            |
| **Self-hosted models**            | Route to Ollama, vLLM, or other servers with [a model you run yourself](/ai-gateway/guides/use-a-model-you-run-yourself) |

## Next steps

<CardGroup cols={2}>
  <Card title="Quickstart" icon="rocket" href="/ai-gateway/quickstart">
    Get ngrok.ai integrated in 5 minutes
  </Card>

  <Card title="Access Keys" icon="key" href="/ai-gateway/concepts/access-keys">
    Learn about managing access to ngrok.ai
  </Card>

  <Card title="How It Works" icon="gears" href="/ai-gateway/how-it-works">
    Request flow and failover behavior
  </Card>

  <Card title="SDK Integration" icon="plug" href="/ai-gateway/sdks">
    Connect your application to ngrok.ai
  </Card>

  <Card title="Bring Your Own Keys" icon="key" href="/ai-gateway/concepts/bring-your-own-keys">
    Use your own provider API keys for providers
  </Card>
</CardGroup>