otari

An OpenAI-compatible LLM gateway you own and run.

Route one endpoint to 40+ providers, issue virtual keys, enforce budgets, and track usage.

Docs · otari.ai · Launch blog · Discord

Otari sits between your applications and model providers. It authenticates requests, resolves provider credentials, enforces budgets before dispatch, and records usage afterwards. Provider calls go through any-llm.

Run Otari standalone with your own database and provider keys, or connect a data plane to otari.ai.

Why Otari

• One OpenAI and Anthropic-compatible endpoint for many providers
• Provider credentials stay behind the gateway
• Revocable API keys with user, workspace, and model scope
• Budget checks before spend and usage records after settlement
• Local routing policies for failover, weighting, and learned selection
• Optional code execution, web search, MCP, inference guardrails, and file understanding

Quickstart

Run an ephemeral standalone gateway with Docker:

docker run --rm -p 8000:8000 \
  -e OTARI_MASTER_KEY=SET_A_MASTER_KEY \
  -e OPENAI_API_KEY=YOUR_OPENAI_KEY \
  -e OTARI_CONFIG_YAML='
default_pricing: true
providers:
  openai:
    api_key: ${OPENAI_API_KEY}
' \
  mzdotai/otari:latest \
  otari serve

On the first empty database, Otari creates an API key and prints it once:

No API keys found. Created bootstrap key for first run. Save this key now:
tk-...

Send a request with that key:

curl http://localhost:8000/api/v1/chat/completions \
  -H "Authorization: Bearer tk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai:gpt-4o-mini",
    "messages": [{"role": "user", "content": "Say hello."}]
  }'

OpenAI clients work by setting base_url to http://localhost:8000/api/v1.

This container uses SQLite inside the container and is deleted when it stops. Use the Compose setup below for persistent data.

The command a developer runs next to a coding agent (otari hook, otari import claude-code) installs on its own, without the server:

brew install mozilla-ai/tap/otari

See Agent Guardrails and Use with Claude Code.

Run the full stack

git clone https://github.com/mozilla-ai/otari
cd otari
cp config.example.yml config.yml
# Set a master key, provider credentials, and pricing in config.yml.
# Provider copies of attached files need a pepper of their own. Compose reads .env.
grep -qs OTARI_PROVIDER_ACCOUNT_PEPPER .env || echo "OTARI_PROVIDER_ACCOUNT_PEPPER=$(openssl rand -base64 32)" >> .env
docker compose pull
docker compose up -d

Compose runs Otari with PostgreSQL. Optional profiles add the code sandbox, web search, and guardrail services:

docker compose --profile code-exec --profile web-search --profile guardrails up -d

The dashboard is served at http://localhost:8000/. To store provider keys through the dashboard, set OTARI_SECRET_KEY to a Fernet key generated by otari gen-secret-key.

A one-click deployment template is available for Railway.

Runtime modes

When OTARI_MODE is unset, OTARI_AI_TOKEN selects hybrid mode; otherwise Otari defaults to standalone. A mode says what a process serves, not who runs it, so "hosted" is not the opposite of "self-hosted". See Runtime modes.

API and dashboard

The core completion routes are:

• POST /api/v1/chat/completions
• POST /api/v1/messages
• POST /api/v1/responses

Standalone also serves the broader OpenAI-compatible and management APIs. The running server publishes Swagger UI at /api/v1/docs and OpenAPI at /api/v1/openapi.json. See API reference.

The dashboard manages providers, models, routing, tools, keys, members, budgets, settings, activity, and usage. Its navigation adapts to the deployment mode and the signed-in person's authority. See Admin dashboard.

Development

git clone https://github.com/mozilla-ai/otari
cd otari
uv sync --dev
cp config.example.yml config.yml
uv run otari serve --config config.yml

For local development without PostgreSQL, change database_url to sqlite+aiosqlite:///./otari.db.

Common checks:

make test
make lint
make typecheck

See CONTRIBUTING.md before opening a pull request.

Documentation

• Quickstart
• Deployment
• Configuration
• Models
• Routing
• Access control
• Built-in tools
• Agent Guardrails
• SDK and agent integrations
• Architecture

License

Apache 2.0. See LICENSE.