Skip to main content
Prerequisite: Complete the Quickstart to get an access key. All requests go to https://gateway.ngrok.ai.
TanStack AI is a type-safe library for building AI applications with React, Solid, and other frameworks. It works with the ngrok AI Gateway through its OpenAI adapter.

​
Installation

npm install @tanstack/ai @tanstack/ai-openai

​
Basic usage

Configure the OpenAI adapter to use your AI Gateway endpoint: 
import { chat } from "@tanstack/ai";
import { openai } from "@tanstack/ai-openai";

const adapter = openai({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

const stream = chat({
  adapter,
  messages: [{ role: "user", content: "Hello!" }],
  model: "gpt-4o",
});

​
Streaming responses

TanStack AI is built for streaming. Process chunks as they arrive: 
import { chat } from "@tanstack/ai";
import { openai } from "@tanstack/ai-openai";

const adapter = openai({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

const stream = chat({
  adapter,
  messages: [{ role: "user", content: "Write a poem about AI gateways." }],
  model: "gpt-4o",
});

for await (const chunk of stream) {
  // Process each chunk as it arrives
  console.log(chunk);
}

​
API route handler

Build API endpoints that stream responses:
app/api/chat/route.ts
import { chat, toStreamResponse } from "@tanstack/ai";
import { openai } from "@tanstack/ai-openai";

const adapter = openai({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

export async function POST(request: Request) {
  const { messages } = await request.json();

  const stream = chat({
    adapter,
    messages,
    model: "gpt-4o",
  });

  return toStreamResponse(stream);
}

​
Using different providers

The AI Gateway routes based on the model name. Use provider prefixes to target specific providers: 
import { chat } from "@tanstack/ai";
import { openai } from "@tanstack/ai-openai";

const gateway = openai({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key // Gateway handles auth
});

// Use different providers through the same gateway
const openaiStream = chat({ adapter: gateway, model: "openai:gpt-4o", messages });
const anthropicStream = chat({ adapter: gateway, model: "anthropic:claude-3-5-sonnet-latest", messages });

​
Model failover

To try another model when the first one fails, see Configure fallback models.

​
Tool calling

The AI Gateway supports function/tool calling: 
import { chat, toolDefinition } from "@tanstack/ai";
import { openai } from "@tanstack/ai-openai";
import { z } from "zod";

const adapter = openai({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

const getWeatherDef = toolDefinition({
  name: "get_weather",
  description: "Get the current weather",
  inputSchema: z.object({
    location: z.string().describe("The location to get weather for"),
  }),
});

const getWeather = getWeatherDef.server(async ({ location }) => {
  return { temperature: 72, conditions: "sunny" };
});

const stream = chat({
  adapter,
  messages: [{ role: "user", content: "What's the weather in San Francisco?" }],
  model: "gpt-4o",
  tools: [getWeather],
});

​
Provider options

Pass provider-specific options for fine-grained control: 
import { chat } from "@tanstack/ai";
import { openai } from "@tanstack/ai-openai";

const adapter = openai({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

const stream = chat({
  adapter,
  messages,
  model: "gpt-4o",
  providerOptions: {
    temperature: 0.7,
    maxTokens: 1000,
    topP: 0.9,
    frequencyPenalty: 0.5,
    presencePenalty: 0.5,
  },
});

​
React integration

Use TanStack AI with React for building chat interfaces:
app/page.tsx
"use client";

import { useChat } from "@tanstack/ai-react";

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: "/api/chat",
  });

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id}>
          {m.role}: {m.content}
        </div>
      ))}
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}

​
Environment variables

Set up your environment:
.env.local
AI_GATEWAY_URL=https://gateway.ngrok.ai/v1
AI_GATEWAY_ACCESS_KEY=ng-xxxxx-g1-xxxxx

import { openai } from "@tanstack/ai-openai";

const adapter = openai({
  baseURL: process.env.AI_GATEWAY_URL,
  apiKey: process.env.AI_GATEWAY_ACCESS_KEY!,
});

​
Next steps