Overview

AI Gateway exposes OpenAI-compatible and Anthropic-compatible APIs. Your application sends requests to AI Gateway with a virtual key and a public model name. AI Gateway selects a provider deployment, translates the request, and tracks usage and cost.

You control the providers, routing rules, and access limits. Supported operations and parameters depend on the deployments configured for each public model.

Start here

Connect an existing application

If an operator has already configured AI Gateway, use its base URL, a virtual key, and an available public model name. For example, with the OpenAI Python SDK:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:4000/v1", api_key="<virtual-key>")
response = client.chat.completions.create(
    model="general",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

Here, general is a name configured in AI Gateway. It can route to one or more upstream models without changing the application's request. See the API reference for authentication, supported endpoints, and errors.

Configure and operate

Contribute

Read Architecture for the request pipeline, Testing for the test workflow, and Documentation development to work on these pages.