Swama

English | 中文 | 日本語

Swama is a local AI runtime for Apple Silicon Macs, written in Swift on Apple's MLX. It runs language, vision, embedding, speech-recognition and text-to-speech models on your Mac and serves them through an OpenAI-compatible API, a command-line tool and a menu bar app.

• OpenAI-compatible API: chat completions (streaming, tool calling, image input), a stateless subset of Responses, embeddings, audio transcription and text-to-speech (experimental).
• Decision scoring: /v1/decisions follows the OpenAI Decisions API and scores predicates, choices, and ratings without generating text.
• Model aliases: swama run qwen3.5 "…" downloads the model from Hugging Face on first use.
• Menu bar app: runs the server in the background, installs the swama command and sets the context limit.

Requirements

• Apple Silicon Mac, macOS 15.4 or later
• Building from source only: Xcode with the Swift 6.2 toolchain

Install

Homebrew

brew install swama

App download: get Swama.dmg from Releases, drag Swama.app into Applications and open it. If macOS blocks the first launch, allow it under System Settings › Privacy & Security. Then choose Install Command Line Tool… in the menu bar to add swama to your PATH.

From source

git clone https://github.com/Trans-N-ai/swama.git
cd swama/swama
swift build -c release
mv .build/release/swama .build/release/swama-bin   # the app bundles the CLI under this name

cd ../swama-macos/Swama
xcodebuild -project Swama.xcodeproj -scheme Swama -configuration Release

Quick start

swama run qwen3.5 "Hello!"                          # downloads on first use
swama run qwen3.5 "What's in this image?" -i photo.jpg
swama serve --host 127.0.0.1 --port 28100          # API server

curl http://localhost:28100/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3.5", "messages": [{"role": "user", "content": "Hello!"}]}'

swama serve binds to 0.0.0.0 by default, which makes the API reachable from other machines on your network. Pass --host 127.0.0.1 to keep it local.

Models

Any MLX model on Hugging Face can be used by its full id (for example mlx-community/Qwen3.5-9B-4bit). Common models also have short aliases:

The complete alias → model mapping is in ModelAliases.swift. FireRedASR2 is also supported by its full repository id. Known issue: in our testing on 2.4.0, gemma3 (4B) and qwen3-vl (4B) fail to load (for Gemma, possibly related to #23); use qwen3.5 for image input.

API

The server listens on port 28100 by default (--port or SWAMA_PORT to change it).

# Image input
curl http://localhost:28100/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "qwen3.5",
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "What do you see?"},
    {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}]}]}'

# Tool calling
curl http://localhost:28100/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "qwen3.5",
  "messages": [{"role": "user", "content": "What is the weather in Tokyo?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather",
    "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}}]}'

# Embeddings
curl http://localhost:28100/v1/embeddings -H "Content-Type: application/json" \
  -d '{"model": "mlx-community/embeddinggemma-300m-4bit", "input": ["Hello world"]}'

# Transcription
curl http://localhost:28100/v1/audio/transcriptions -F "file=@audio.wav" -F "model=qwen3-asr"

# Text-to-speech
curl http://localhost:28100/v1/audio/speech -H "Content-Type: application/json" \
  -d '{"model": "kokoro", "input": "Hello from Swama", "response_format": "wav"}' --output speech.wav

Voices: Orpheus dan jess leo mia tara zac zoe; Marvis conversational_a conversational_b; Qwen3-TTS and VyvoTTS en-us-1; Kokoro defaults to af_heart, KittenTTS to Bella.

/v1/responses support matrix

POST /v1/responses implements an honest, stateless subset of the OpenAI Responses API:

• Supported: string or message-item input, instructions, input_text and input_image parts, custom function tools including multi-turn function_call / function_call_output items, tool_choice "auto"/"none", basic sampling (temperature, top_p, max_output_tokens), non-streaming Response objects, and typed SSE streaming with monotonic sequence_number.
• Accepted with an explicit local meaning (the fixed envelope Codex CLI sends — validated, documented, never silently honoured):
  • client_metadata — client-side tracing only; validated and ignored.
  • prompt_cache_key — accepted as a hint and currently not used. Swama's local cache reuses a KV prefix by comparing actual prompt tokens, and holds one entry per model, so sessions interleaved against the same model evict each other. This is not an equivalent of the hosted prompt cache.
  • reasoning — the empty object, or summary: "auto" (what Codex sends). auto leaves the choice to the server and a local model emits no reasoning items, so producing none satisfies it.
  • include: ["reasoning.encrypted_content"] — accepted; there are no reasoning items to return.
  • parallel_tool_calls — this server never executes tools and always emits function-call items one after another, satisfying false.
  • tools[{"type":"web_search","external_web_access":false}] — accepted and then dropped before the model sees it. Codex CLI 0.147.0 advertises this tool unconditionally and no client setting removes it (tools.web_search = false only flips external_web_access), so refusing it would make the CLI unusable against Swama. This is a deliberate, narrow compatibility degradation, not support: per OpenAI's reference, external_web_access: false does not disable search — it runs web search in an offline, cache-only mode over OpenAI's own index. Swama has no such index, performs no search whatsoever, and never emits a web_search_call. Any request that genuinely needs search capability is not supported here. true, a missing or non-boolean flag, any additional key (filters, search_context_size, ...), and a tool_choice naming web_search all remain hard 400s.
• Rejected with an explicit 400 (never silently ignored): server-side state (store: true, previous_response_id, conversation, prompt), background: true, hosted/built-in and MCP tools, Structured Outputs (text.format other than plain text, response_format), truncation: "auto", forced tool_choice, reasoning.effort and any reasoning.summary other than "auto", any other include entry, max_tool_calls, service_tier, text.verbosity, tools[].strict: true, hosted tool kinds other than the exact offline web_search shape above (including namespace — run Codex with --disable multi_agent), image URLs outside http/https/data:image/..., every unlisted top-level field, and wrong-typed known fields.
• Responses are not stored: there is no retrieve/cancel/delete by response id.

/v1/decisions (OpenAI Decisions API)

POST /v1/decisions follows the OpenAI Decisions API (openai-openapi 4a4020d8). It scores a finite set of answers without generating text. Breaking change: the earlier SGLang prompt format 1 wire (questions with id/question/options, yes_no, label_mass, temperature, chat_template_kwargs, prompt_format_version, return_prompt_token_ids) has been removed from this route; such requests now get HTTP 400 with a message saying the SGLang prompt format 1 request shape was removed (a question with id, question, options or type yes_no); a removed top-level field alone, such as temperature on an otherwise OpenAI-shaped request, is reported as an unknown field. /v1/systemone is unchanged.

Request. model (an explicit local model), input, questions (1–200) and safety_identifier (a string of at most 128 characters, or null; accepted and ignored). Parsing is strict: an unknown field at any level is refused with HTTP 400.

• predicate: {"type":"predicate","name"?,"instructions"}, the probability that a statement about the input is true.
• choice: {"type":"choice","name"?,"instructions","choices":[{"value","description"?}]} with 2–255 choices. A value is a string or a boolean and comes back with the same JSON type.
• score: {"type":"score","name"?,"instructions","levels":[{"label","description"?}]} with 2–10 ordered levels.

instructions must not be blank, and a blank score level label is refused (stricter than the OpenAI schema). name is optional and must be unique when given. Duplicate choice values are refused. The prompt shows a boolean as true/false, so a string "true" and a boolean true in the same choice cannot be told apart and are refused too. Choice values must otherwise be non-empty, contain no control characters and be distinct ignoring case. A level with a description is shown as label: description. More than 26 choices use the model's two-letter labels (AA, AB, …), as on /v1/systemone; a model without enough single-token pair labels refuses them.

input is a string, or an array of user messages {"role":"user","content":…,"type"?:"message"} whose content is a string or a list of input_text and input_image parts. Texts are joined with newlines in order and images keep their order. Images must be base64 data URLs and follow the /v1/systemone image rules below: PNG, JPEG or WebP checked by file signature, complete files, at most 4 MiB and 16 megapixels each, 8 MiB in total, and at most 4 per request (fewer than OpenAI's 128). They need a vision model. When every image sets detail: "low", images are resized to fit 512 pixels per side, which is several times faster (Qwen3.5-9B on an M5 Pro: one 1280-pixel photo drops from about 1,240 to 240 input tokens and from 1.8 s to 0.36 s); otherwise detail keeps the default resizing. Other roles, other part types and remote URLs are refused. A blank input is allowed and rendered as is.

Response. model, answers in question order, and usage. Each answer carries its name, or null when the question had none.

• predicate: probability, the yes probability conditional on the yes/no labels.
• choice: choice (the typed value with the highest probability; ties go to the first), probabilities as [{value, probability}] in request order, and confidence.
• score: score (the expected level index), probabilities as [{value, label, probability}] with an integer value, and confidence.

Answers are never refusal; a request the backend cannot serve is refused as a whole with a 4xx error ({"error":{"message","type"}}; unknown models use 404). usage reports input_tokens and total_tokens; output, cached, cache-write and reasoning tokens are 0. Temperature is fixed at 1.

How it scores. Swama renders one prompt per question and reads the next-token probabilities of the answer labels (yes/no, A, B, …, or 0, 1, …), using SGLang's public prompt wording. Probabilities are normalized over those labels only; the probability left outside them is not reported. The local chat tokenizer must preserve the rendered prompt and encode every answer label as one distinct token at the answer position. The template gets enable_thinking: false; templates may ignore it. Open reasoning prefixes and a recognized single-token `` or [THINK] opener at the vocabulary maximum are rejected. This check does not certify every reasoning format or guarantee the model follows instructions. Each question starts with a fresh KV cache, independent of the chat prompt cache. Even identical low-precision weights can produce probability differences across backends and prefill layouts.

confidence uses the formulas from SGLang's SystemOne implementation. It measures concentration among the candidates, not the probability that the answer is correct. Let q be the candidate probabilities, n the number of candidates, and m the first index with maximum probability (levels in request order):

• Choice: clamp((n * max(q) - 1) / (n - 1), 0, 1).
• Score: max(0, 1 - sum(q[i] * abs(i - m)) / U), where U = sum(abs(i - (n - 1) / 2)) / n. This uses absolute distance and the uniform distribution's midpoint, not variance or a denominator centered on m.

For example, three choice probabilities [0.7, 0.2, 0.1] give confidence 0.55; uniform probabilities give 0, and a one-hot distribution gives 1. Validate thresholds on representative application data; confidence does not guarantee correctness.

Known limitations

• predicate reads only lowercase labels. As in SGLang, only the lowercase yes and no tokens are scored. Many models also put probability on Yes and No. When a model prefers the capitalized form for one answer but not the other, probability can differ from the case-combined answer, and in independent testing it occasionally pointed the other way. Swama does not merge case variants; validate predicate thresholds on your own data.
• The measured Qwen3.5 models produce bf16 logits. Two labels can tie exactly; ties go to the first option in the order given. Values can shift between Swama versions, dependency updates and backends; do not rely on bit-for-bit probability agreement across those configurations.
• Mixture-of-experts models drift more. Against a full-precision (fp32) computation of the same 4-bit weights, centered label log-probabilities differed by up to about 0.2 for dense Qwen3.5 models (0.8B, 9B) and up to about 0.55 for Qwen3.5-35B-A3B. The top answer matched the full-precision result in all 21 reference cases we measured, but close calls on MoE models are less stable. Whether that is acceptable depends on your application.

curl -X POST http://localhost:28100/v1/decisions \
  -H "Content-Type: application/json" \
  -d '{"model":"mlx-community/Qwen3.5-0.8B-4bit","input":"My invoice charged me twice.","questions":[{"type":"predicate","name":"billing","instructions":"Is this a billing issue?"},{"type":"choice","name":"team","instructions":"Which team should handle this?","choices":[{"value":"billing"},{"value":"technical"},{"value":"sales"}]},{"type":"score","name":"urgency","instructions":"How urgent is it?","levels":[{"label":"low"},{"label":"medium"},{"label":"high","description":"needs action today"}]},{"type":"choice","instructions":"Was the customer charged correctly?","choices":[{"value":true},{"value":false}]}]}'

Response from that model (formatted):

{"model":"mlx-community/Qwen3.5-0.8B-4bit","answers":[
  {"type":"predicate","name":"billing","probability":0.8175744761936437},
  {"type":"choice","name":"team","choice":"billing","probabilities":[{"value":"billing","probability":0.9478362250325003},{"value":"technical","probability":0.047189986937231164},{"value":"sales","probability":0.004973788030268673}],"confidence":0.9217543375487502},
  {"type":"score","name":"urgency","score":1.922966354728045,"probabilities":[{"value":0,"label":"low","probability":0.015163456120649954},{"value":1,"label":"medium","probability":0.04670673303065524},{"value":2,"label":"high","probability":0.9381298108486948}],"confidence":0.8844495320920672},
  {"type":"choice","name":null,"choice":true,"probabilities":[{"value":true,"probability":0.5621765008857981},{"value":false,"probability":0.4378234991142019}],"confidence":0.12435300177159614}],
 "usage":{"input_tokens":184,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":184}}

/v1/systemone (SystemOne OpenAPI 0.2.0)

POST /v1/systemone accepts an explicit local model, textual or structured state, and a map of named questions. It calls the same engine, model pool, prompt wording and scorer as /v1/decisions. noul maps to yes/no and returns noul = p(yes); choice returns the selected name, probabilities and confidence; score returns the expected level, probabilities, confidence and the original criteria in legend. The response has model, answers and usage (input_tokens, output_tokens: 0). x_label_mass is the total probability of the answer labels against the full vocabulary; a low value means the model may prefer an answer outside the requested set. Confidence remains concentration, not calibrated correctness. The model/template and lowercase yes/no limitations of /v1/decisions above also apply.

Swama's current backend accepts 2–26 choices, 2–10 score levels, and non-empty instructions. Single-option questions, empty or omitted instructions, invalid rubrics, and unsupported counts receive a readable 4xx refusal. No question text or probability is invented to work around a refusal. Validation errors use HTTP 400; unknown models use 404. Capacity refusals include the phrases recognized by Decision Index's HTTP engine. Duplicate JSON object keys are rejected instead of overwriting the earlier value.

Question and choice-criteria maps retain request order; that order determines labels and the first winner on ties. String state is passed through. Objects and arrays render as compact Unicode JSON in their original member order, following SGLang render_text (openai/serving_decisions.py:369–374, frozen commit eb9c9ee9). For an exact comparison, send that same rendered text as the Decisions input string. Rubric descriptions may be strings, objects or arrays; the score answer echoes their original JSON values in legend. Temperature, prompt version and token-ID-return fields are refused on this route. chat_template_kwargs supports only enable_thinking: false.

The official Python SDK and JavaScript SDK can call this endpoint with a local base URL and model. An SDK API key placeholder is needed by their constructors; Swama does not use it for authentication. Model discovery keeps the OpenAI-shaped /v1/models. Pass an explicit local model name to TypeSafe SDK calls; its client.models.list() expects a different catalog format.

Images (Cloudflare Clef extension). An optional top-level images array follows Clef's System One extension: up to 4 images placed before the state, each either a base64 data URL string ("data:image/png;base64,...") or an object {"content_type": "image/png", "base64": "..."}. Only PNG, JPEG and WebP are accepted, judged by the file signature; a declared type that does not match the bytes is refused, and so is a truncated file. Limits: 4 MiB and 16 megapixels per image (read from the header before decoding), 8 MiB decoded in total, and a 13 MiB request body (HTTP 413). Remote URLs and video are refused. Images need a vision model and share the chat path's multimodal context limit (4096 tokens). Images are resized the way the chat path resizes them (up to 1344 px for Qwen3.5), so in practice Qwen3.5-9B fits at most 3 images per request; a request over the limit is refused with maximum context length. "images": [] is the same as no field. /v1/decisions takes images only in the OpenAI format, as input_image parts.

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

with TypeSafeClient(
    base_url="http://127.0.0.1:28100",
    api_key="local",
    model="mlx-community/Qwen3.5-0.8B-MLX-4bit",
) as client:
    result = client.system_one(
        state={"document": "I was charged twice. Please help."},
        questions={
            "team": Choice(instructions="Which team?", criteria={"billing": None, "technical": None}),
            "billing": Noul(instructions="Is this a billing issue?"),
            "urgency": Score(instructions="How urgent?", criteria=["Can wait", "Needs attention today"]),
        },
    )
    print(result.choices["team"].choice)

Command line

run and serve accept --context-limit (default 16384 tokens).

Environment variables: SWAMA_PORT (server port), SWAMA_MODELS (model directory, default ~/.swama/models), SWAMA_CONTEXT_LIMIT, SWAMA_REGISTRY (HUGGING_FACE or MODEL_SCOPE), SWAMA_PROMPT_CACHE=0 (disable prompt caching), SWAMA_DIAGNOSTICS_PATH (diagnostics log location), SWAMA_DIAGNOSTICS_DISABLED=1 (turn the log off).

Development

The Swift package is in swama/ (swift build, swift test) and the macOS app in swama-macos/. See CONTRIBUTING.md for the workflow and SECURITY.md for reporting vulnerabilities.

Built on mlx-swift, mlx-swift-lm, mlx-audio-swift, swift-transformers, swift-nio and swift-argument-parser.

License

MIT — see LICENSE. Questions and bug reports: Issues · Discussions.