← Open Source
magnitudedev

magnitude

Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU.

AI EngineeringGive agent toolsBuild agent workflow (SDK)Rust
Open on GitHub
Momentum
+15stars in 24 hours+0.2%
7.91k
Stars
574
Forks
+119
This week
13
Contributors
Created 2026-06-12 · Updated 2026-11-19 · #1001 today
Top developers
README

Magnitude

Run open models as fast as your hardware allows

Magnitude is an open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. It's over 2x faster than Ollama and works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU. One click connects it to the agent you already use (Pi, OpenCode, Hermes, Codex, and more).

⭐ Help us reach more developers and grow the Magnitude community. Star this repo!

Get started

Magnitude comes with a desktop app and a CLI. Install from your terminal:

macOS / Linux:

curl -fsSL https://magnitude.dev/install.sh | sh

Windows PowerShell:

irm https://magnitude.dev/install.ps1 | iex

Or download the installer:

To run Magnitude as a service other computers can use, see Remote server. To run it in a container, see Docker.

Why Magnitude?

  • 2x faster than Ollama: on average across popular local models, on Mac and NVIDIA
  • Tuned on your hardware: kernels are tuned for each model when you download it
  • Best drafter built in: each model ships with the fastest speculative decoding drafter
  • Memory that flexes: quantized KV halves memory per agent, freed when agents stop
  • Fast concurrent sessions: sessions share prefix caches to prevent slowdown
  • Works with your agent: one click to connect Pi, OpenCode, Hermes, Codex, and more
  • Free, private, open source: no token costs, nothing leaves your machine, Apache 2.0

Over 2x faster than Ollama

View full benchmarks →

FAQ

What is Magnitude?

An open source inference engine that optimizes each model for your exact hardware. It ships as a desktop app that runs open models and connects them to the agent you already use.

How is it faster than llama.cpp, Ollama, or LM Studio?

They ship one-size-fits-all kernels, because tuning each model for each chip by hand would take forever. Magnitude tunes them automatically. When you download a model, it spends up to a minute on your actual chip trying different ways to split up the math (how many GPU threads team up, how big a chunk each one takes, how far ahead to load data) and keeps the fastest combination that doesn't change the model's output. See the full benchmarks against Ollama.

What hardware do I need?

Any Apple Silicon, NVIDIA, AMD, or Intel GPU, or nothing but a CPU. There is no fixed minimum. Smaller machines run smaller models, and more memory lets you run larger ones.

What operating systems does it support?

macOS, Linux, and Windows.

Which models does it support?

See the full list at magnitude.dev/models. We write optimized kernels for the most popular open-weight families, which is how we beat generalist engines.

Which agents work with it?

One click connects Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. Anything else works through the OpenAI-compatible API.

Is it private?

Yes. Prompts, files, and models stay on your machine. No internet needed once a model is downloaded.

Nearby in the ranking
Similar items

AI Rank for iPhone

Follow AI model rankings

Download on the App Store