otari · mozilla-ai/otari GitHub 开源项目

Open-source, OpenAI-compatible LLM gateway you run yourself. One endpoint for 40+ providers, with virtual keys, budgets, and usage tracking.

An OpenAI-compatible LLM gateway you own and run.

Route one endpoint to 40+ providers, issue virtual keys, enforce budgets, and track usage.

Docs · otari.ai · Launch blog · Discord

Otari sits between your applications and model providers. It authenticates requests, resolves provider credentials, enforces budgets before dispatch, and records usage afterwards. Provider calls go through any-llm.

Run Otari standalone with your own database and provider keys, or connect a data plane to otari.ai.

Why Otari

One OpenAI and Anthropic-compatible endpoint for many providers
Provider credentials stay behind the gateway
Revocable API keys with user, workspace, and model scope
Budget checks before spend and usage records after settlement
Local routing policies for failover, weighting, and learned selection
Optional code execution, web search, MCP, inference guardrails, and file understanding

Quickstart

Run an ephemeral standalone gateway with Docker:

docker run --rm -p 8000:8000 \
  -e OTARI_MASTER_KEY=SET_A_MASTER_KEY \
  -e OPENAI_API_KEY=YOUR_OPENAI_KEY \
  -e OTARI_CONFIG_YAML='
default_pricing: true
providers:
  openai:
    api_key: ${OPENAI_API_KEY}
' \
  mzdotai/otari:latest \
  otari serve

On the first empty database, Otari creates an API key and prints it once:

No API keys found. Created bootstrap key for first run. Save this key now:
tk-...

Send a request with that key:

curl http://localhost:8000/api/v1/chat/completions \
  -H "Authorization: Bearer tk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai:gpt-4o-mini",
    "messages": [{"role": "user", "content": "Say hello."}]
  }'

OpenAI clients work by setting base_url to http://localhost:8000/api/v1.

This container uses SQLite inside the container and is deleted when it stops. Use the Compose setup below for persistent data.

The command a developer runs next to a coding agent (otari hook, otari import claude-code) installs on its own, without the server:

brew install mozilla-ai/tap/otari

See Agent Guardrails and Use with Claude Code.

Run the full stack

git clone https://github.com/mozilla-ai/otari
cd otari
cp config.example.yml config.yml
# Set a master key, provider credentials, and pricing in config.yml.
# Provider copies of attached files need a pepper of their own. Compose reads .env.
grep -qs OTARI_PROVIDER_ACCOUNT_PEPPER .env || echo "OTARI_PROVIDER_ACCOUNT_PEPPER=$(openssl rand -base64 32)" >> .env
docker compose pull
docker compose up -d

Compose runs Otari with PostgreSQL. Optional profiles add the code sandbox, web search, and guardrail services:

docker compose --profile code-exec --profile web-search --profile guardrails up -d

The dashboard is served at http://localhost:8000/. To store provider keys through the dashboard, set OTARI_SECRET_KEY to a Fernet key generated by otari gen-secret-key.

A one-click deployment template is available for Railway.

Runtime modes

Mode	Purpose
Standalone	One process serves management and inference using local storage.
Hosted	A multi-tenant control plane; inference runs on connected gateways.
Hybrid	A data-plane gateway resolves credentials and reports usage to otari.ai.

When OTARI_MODE is unset, OTARI_AI_TOKEN selects hybrid mode; otherwise Otari defaults to standalone. A mode says what a process serves, not who runs it, so "hosted" is not the opposite of "self-hosted". See Runtime modes.

API and dashboard

The core completion routes are:

POST /api/v1/chat/completions
POST /api/v1/messages
POST /api/v1/responses

Standalone also serves the broader OpenAI-compatible and management APIs. The running server publishes Swagger UI at /api/v1/docs and OpenAPI at /api/v1/openapi.json. See API reference.

The dashboard manages providers, models, routing, tools, keys, members, budgets, settings, activity, and usage. Its navigation adapts to the deployment mode and the signed-in person's authority. See Admin dashboard.

Development

git clone https://github.com/mozilla-ai/otari
cd otari
uv sync --dev
cp config.example.yml config.yml
uv run otari serve --config config.yml

For local development without PostgreSQL, change database_url to sqlite+aiosqlite:///./otari.db.

Common checks:

make test
make lint
make typecheck

See CONTRIBUTING.md before opening a pull request.

Documentation

License

Apache 2.0. See LICENSE.