Skip to main content
Azure OpenAI Service provides OpenAI models on Microsoft Azure. Connect your deployment to the AI Gateway as a custom provider.

​
What you’ll need

​
Overview

Azure OpenAI uses deployment-specific URLs and requires an api-version query parameter. The gateway injects your Azure API key server-side.

​
Getting started

1

Gather Azure details

From the Azure Portal, collect:
  1. Resource endpoint: https://your-resource.openai.azure.com
  2. API key: from Keys and Endpoint
  3. Deployment name: the name of your model deployment
  4. API version: a supported version (for example 2024-02-15-preview)
2

Create the custom provider

Azure requires the deployment path and api-version in the base URL. See Create a custom provider with:
  • Provider ID: azure-openai
  • Base URL: https://your-resource.openai.azure.com/openai/deployments/your-deployment?api-version=2024-02-15-preview
  • API format: OpenAI Chat Completions
  • Models: the model name for this deployment (for example gpt-4o)
Include api-version as a query parameter on the base URL. Azure rejects requests without it.
3

Store your Azure API key

Add a provider key for your Azure deployment.
4

Configure access

Create an access key configuration that:
  1. Allows the azure-openai provider in the access scope
  2. Adds a routing rule for azure-openai with Bring your own API key and your stored key attached
Assign the configuration to your access key.
5

Send requests

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx"
)

response = client.chat.completions.create(
    model="azure-openai:gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

​
Multiple deployments and regions

Create a separate custom provider for each Azure deployment or region. Give each a unique provider ID (for example azure-gpt4, azure-eastus) with its own deployment URL and model list. Attach the same or different provider keys per deployment. Pin a region with azure-eastus:gpt-4o, or list multiple models for failover: models: ["azure-eastus:gpt-4o", "openai:gpt-4o"].

​
Troubleshooting

SymptomFix
401 unauthorizedVerify the provider key in app.ngrok.ai matches Azure Portal and is attached in your access key configuration
404 deployment not foundCheck the deployment name in the base URL matches Azure Portal
API version errorsUpdate the api-version query parameter to a supported version
429 rate limitingAdd multiple provider keys for failover or request a quota increase in Azure Portal

​
Next steps