ZINC

Run open-weight models on the GPU you already own. ZINC is a single Zig binary — no Python, no CUDA-only assumptions — that loads GGUF files and gives you a command line, a browser chat, a model manager, and an OpenAI-compatible API.

Get started · See every benchmark · Read the docs · Join Discord

Speed, measured honestly

Each model below was run on the same Radeon AI PRO R9700 through ZINC's ROCm backend and through llama.cpp's ROCm (HIP) backend, built from the same llama.cpp commit as our Vulkan comparison. Both engines load the same GGUF file, read the same prompt tokens (the harness checks the counts match), run in a reusable server, and get the same warmups and repeat counts. The bars show how ZINC compares on the two things you feel while using a model: how fast it reads your prompt, and how fast it writes the answer.

Reading that honestly:

• Prompt processing is ahead in all 24 rows: 1.07x to 1.9x on the short chat prompt shown here, narrowing to 1.02x-1.31x on the longest prompts.
• Token generation is ahead in all 24 rows, from 1.02x (Gemma 4 31B) to 1.28x (Qwen 3.6 35B-A3B).
• Qwen 3.8 can speculate, and the chart does not count it. That model ships an extra "NextN" block, and ZINC uses it to draft tokens the full model then verifies in one batched pass: identical output, fewer passes, 1.9x the generation speed. llama.cpp can speculate with the same block (its converter exports it as a separate draft model), and our runs do not give it one, so that number would not be a fair comparison. The chart therefore shows both engines without speculation; the benchmark page has a switch for the speculative figure.
• On AMD, pick the backend for the job. ZINC's ROCm backend reads prompts much faster than its Vulkan backend; for generation the two are close, and Vulkan is ahead on some models (Qwen 3.6 35B-A3B: 105 tok/s on Vulkan, 97 on ROCm). The Vulkan build is compared against llama.cpp's Vulkan backend and published separately on the benchmark page.

This is one GPU and six models, not a universal claim. Other cards are measured separately, and rows where llama.cpp is ahead stay on the page.

Every number comes from tools/performance_suite.mjs and lands in site/src/data/zinc-performance.json, recording prompts, raw samples, build revisions and the llama.cpp commit used. The chart above is regenerated from that file, never hand-edited.

Get running

ZINC needs Zig 0.15.2 or newer. Linux Vulkan builds also need glslc and a Vulkan loader; ROCm builds need a working ROCm installation.

git clone https://github.com/zolotukhin/zinc.git
cd zinc
zig build -Doptimize=ReleaseFast

./zig-out/bin/zinc --check                      # what GPU did it find?
./zig-out/bin/zinc model pull qwen35-9b-q4k-m   # fetch a model
./zig-out/bin/zinc --model-id qwen35-9b-q4k-m --prompt "Hello" --chat

On AMD you can pick either backend. Vulkan is the portable default; ROCm uses HIP with kernels tuned for RDNA4 and leads on prompt processing:

ROCM_PATH=/opt/rocm zig build -Dbackend=rocm -Doptimize=ReleaseFast
ROCR_VISIBLE_DEVICES=0 ./zig-out/bin/zinc --check

See Getting started for package prerequisites and the first-run walkthrough, or the dedicated ROCm setup guide.

Chat and API

./zig-out/bin/zinc chat --model-id qwen35-9b-q4k-m

That one command starts the browser chat and an OpenAI-compatible API on the same port, so existing clients and SDKs work unchanged: /health for liveness, /v1/models and /v1/chat/completions for the API. The API guide has curl and SDK examples.

GPU support

Each backend has its own kernels and is benchmarked separately — no backend inherits another's results. The hardware guide lists validated cards, drivers, memory requirements and current limitations.

Models

ZINC reads local GGUF files and ships a managed catalog you can pull from. Current tuning work covers Qwen 3.5, Qwen 3.6, Qwen 3.8, Gemma 4 and Muse Glimmer, including mixture-of-experts and hybrid-attention architectures.

./zig-out/bin/zinc -m /path/to/model.gguf --prompt "The capital of France is"
./zig-out/bin/zinc -hf Qwen/Qwen3-0.6B-GGUF:Q8_0 --prompt "Hello" --chat

The Muse checkpoint used in ZINC measurements is the exact Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf file published by Meta.

Build, test, contribute

zig build -Doptimize=ReleaseFast
zig build test

Start with the development guide and the contributing guide. To reproduce or extend the benchmarks, docs/BENCHMARKING.md documents the suite, including how to publish a speculative-decoding on/off pair.

ZINC is active engineering work. When a model or GPU path is incomplete, the benchmark page leaves that result visible instead of quietly dropping it.

MIT licensed.