Skip to main content
Prerequisite: Complete the Quickstart to get an access key. All requests go to https://gateway.ngrok.ai.
The Vercel AI SDK provides a unified interface for building AI applications. It works with the ngrok AI Gateway, adding provider failover, key rotation, and observability to Vercel’s streaming and UI components.

​
Installation

npm install ai @ai-sdk/openai

​
Basic usage

Point the OpenAI provider at your AI Gateway endpoint: 
import { generateText } from "ai";
import { createOpenAI } from "@ai-sdk/openai";

const openai = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

const { text } = await generateText({
  model: openai("gpt-4o"),
  prompt: "What is the meaning of life?",
});

​
Streaming responses

The AI Gateway supports streaming. Use streamText for real-time responses: 
import { streamText } from "ai";
import { createOpenAI } from "@ai-sdk/openai";

const openai = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

const result = streamText({
  model: openai("gpt-4o"),
  prompt: "Write a poem about AI gateways.",
});

for await (const textPart of result.textStream) {
  process.stdout.write(textPart);
}

​
Chat interface

Build chat applications with the useChat hook:
app/page.tsx
"use client";

import { useChat } from "ai/react";

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: "/api/chat",
  });

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id}>
          {m.role}: {m.content}
        </div>
      ))}
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
app/api/chat/route.ts
import { streamText } from "ai";
import { createOpenAI } from "@ai-sdk/openai";

const openai = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai("gpt-4o"),
    messages,
  });

  return result.toDataStreamResponse();
}

​
Using different providers

The AI Gateway routes based on the model name. Use provider prefixes to be explicit: 
import { generateText } from "ai";
import { createOpenAI } from "@ai-sdk/openai";

const gateway = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

// Use different providers through the same gateway
const openaiResult = await generateText({ model: gateway("openai:gpt-4o"), prompt: "Hello" });
const anthropicResult = await generateText({ model: gateway("anthropic:claude-3-5-sonnet-latest"), prompt: "Hello" });

​
Model failover

List fallback models in the request. See Configure fallback models.

​
Environment variables

Set up your environment:
.env.local
AI_GATEWAY_URL=https://gateway.ngrok.ai/v1
AI_GATEWAY_ACCESS_KEY=ng-xxxxx-g1-xxxxx

const openai = createOpenAI({
  baseURL: process.env.AI_GATEWAY_URL,
  apiKey: process.env.AI_GATEWAY_ACCESS_KEY!,
});

​
Tool calling

The AI Gateway supports function/tool calling: 
import { generateText, tool } from "ai";
import { createOpenAI } from "@ai-sdk/openai";
import { z } from "zod";

const openai = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

const { text, toolCalls } = await generateText({
  model: openai("gpt-4o"),
  tools: {
    weather: tool({
      description: "Get the weather in a location",
      parameters: z.object({
        location: z.string().describe("The location to get weather for"),
      }),
      execute: async ({ location }) => {
        return { temperature: 72, condition: "sunny" };
      },
    }),
  },
  prompt: "What's the weather in San Francisco?",
});

​
Error handling

Handle errors gracefully: 
import { generateText, APICallError } from "ai";

try {
  const { text } = await generateText({
    model: openai("gpt-4o"),
    prompt: "Hello",
  });
} catch (error) {
  if (error instanceof APICallError) {
    console.error("API Error:", error.message);
    console.error("Status:", error.statusCode);
  }
}
With the AI Gateway’s failover, many errors are handled automatically by retrying with different providers or keys.

​
Next steps