project-maya

Project Maya

A 321-billion-parameter AI model on your own GPU

GLM-5.3-Flash, private and fast · one NVIDIA GPU or up to 16 · AMD and Windows (experimental) · chat, pictures, OpenAI- and Anthropic-compatible API · several requests at once

Created and led by Peasant Smith (@mw00), co-maintained with @needmorevram

☕ Support Project Maya - buy me a coffee

Models this size usually need a datacenter. Maya runs GLM-5.3-Flash (zai-org, MIT license) on the hardware you already have: 321 B parameters, about 18 B active per token, and a context of up to 1 M tokens. Its engine is built for this one model. It keeps the experts your conversation uses most on the GPU, the next ones in RAM and the rest on your NVMe SSD, and moves them as you chat. It measures your GPUs, CPU, RAM and SSD and tunes itself to them. Nothing leaves your machine.

• Fast: up to 118 tokens/s on four RTX 4090s, 81 on one RTX PRO 6000, 30+ on a single RTX 5090. Real numbers from real users below.
• Close to the original: Maya's own quants keep 97.7-99.2% of the FP8 model's zero-shot accuracy.
• Plugs into your tools: OpenAI- and Anthropic-compatible API with tool calls, ready for coding agents.
• Several requests at once: up to 8 answered together - for agents that run several sessions, or several people on one PC.
• Everything in the box: a chat and a live monitor in the browser, pictures, thinking levels, a one-command setup.

The models

Project Maya's own quants of GLM-5.3-Flash, made from Z.ai's FP8 release with statistics from the FP8 model. Each keeps the model's MTP block, which drafts tokens ahead.

The setup asks which one to download and recommends one for your PC. How they're made and measured: MAYA-S, MAYA-S24, MAYA-M, MAYA-L. On Hugging Face.

Community quants that also run on Maya

How fast is it?

Real machines, real numbers, most of them measured with ./maya.sh --bench. A token is about ¾ of a word.

Prefill is an 8K-token prompt. The speed holds as a conversation grows, and the first answers after a start are the slowest while the caches fill with what you use. Share yours: --bench, then --report, in a GitHub issue.

Several requests at once (--parallel) add up: on one GPU a token of each goes through the model together, so the dense weights are read once for all of them. In tokens per second for all of them together, against the same questions one after another: x1.41 for two and x1.35 for four on a V100 that streams most of Maya-S's experts from RAM, x1.67 for two and x2.02 for four on a Gorgon Halo (Radeon 8065S, 160 GB for the GPU) that holds every expert in GPU memory. The more of the model VRAM holds, the more it gains (how).

What you need

The setup checks all of this, prints the exact command for anything missing, and installs nothing system-wide.

Install

git clone https://github.com/mw00/project-maya.git && cd project-maya
./setup.sh

Answer a few questions (Enter takes the recommended choice). The setup shows every download before it starts it, builds everything once (20-40 minutes plus the download) and starts the model. Then open http://127.0.0.1:8080. Ctrl+C stops it, and ./setup.sh starts it again right away.

• Windows: START-MAYA.bat instead of ./setup.sh - see docs/WINDOWS.md.
• AMD: docs/AMD_MAYA.md.
• Updating: the dashboard's About > Update, or git pull then ./setup.sh. The model isn't downloaded again.
• Every option (GPUs, context, a model folder you already have, remote access, ...): docs/SETUP.md.

The start takes a few minutes: the engine pins most of the free RAM (all but about 6 GB) for its expert tier, so other programs get little RAM while Maya runs.

Using it

• In the browser: chat (kept in your browser, per-chat instructions, edit and regenerate, HTML previews), a live Monitor of the model and your hardware (down to what fills each GPU's memory), and a context size you can change in Settings.
• Your apps and coding agents: an OpenAI-compatible API at http://127.0.0.1:8080/v1, and Anthropic's at http://127.0.0.1:8080/v1/messages (Claude Code: ANTHROPIC_BASE_URL=http://127.0.0.1:8080). Tool calls are streamed as the model writes them, and a long prompt's reading progress too when a request asks for it ("return_progress": true). A web page served from another address can call it too once you allow its address ("cors_origins", docs/SETUP.md).
• Thinking: Off, Low, High (the default) or Max, in Settings or per request.
• Pictures: attach one in the chat, or send image parts through either API.
• From another device: ./maya.sh --setup --host 0.0.0.0 --api-key . Always set a key.
• Tuning: ./maya.sh --calibrate tunes the engine to your hardware in 10-15 minutes. Every setting: docs/SETTINGS.md.
• Several requests at once: ./maya.sh --parallel 4 (Windows: START-MAYA.bat --parallel 4; up to 8) keeps it in the config, and the dashboard shows how many answers are being written. Each extra request keeps its own context in VRAM, so a few fewer experts fit there (how).
• Long contexts: new setups keep each attention layer's cache in RAM with a window of it in VRAM (--kv-streaming on; ./maya.sh --setup adds it to an existing config), so the VRAM holds more experts - at 512K on an RTX 3090, prompts 4-9x faster and decode +5%; 1M context fits one 3090 (how).
• Faster answers with a DFlash2 drafter: ./maya.sh --setup --draft dflash2 (NVIDIA RTX 30 / A100 and newer) downloads a small drafter that drafts up to 7 tokens a round and gives the same answers - Maya-L +44% on nine GPUs and +12% on two (against the MTP block), +19% on one RTX 3090 (how). How much it gains depends on the PC: on one GPU that reads much of the model from RAM it is often slower. So a speed calibration measures it on your PC and keeps it only where it is faster. The drafter's license is non-commercial (CC BY-NC-ND 4.0).

Something went wrong?

Run ./maya.sh --report (Windows: START-MAYA.bat --report) right after a slow answer or an error, and attach the maya-report.txt it writes to a GitHub issue. It holds your hardware, your setup and the engine's speed lines; nothing is sent anywhere by itself. The engine's own log is maya-.log in the Maya folder.

• "nvcc ... cannot build for these GPUs": Volta needs CUDA 12.x, Blackwell 12.8 or newer. The setup takes the newest installed toolkit that fits.

• cuda_profiler_api.h or nvtx3/nvToolsExt.h is missing: Maya's CUDA build needs the profiler API and NVTX headers as well as nvcc and the runtime. Install those development components for your toolkit. With a split CUDA 13 toolkit, NVIDIA's official header wheels can also stay in a project-local folder:

  python3 -m pip install --target .cuda-headers \
    nvidia-cuda-profiler-api==13.3.27 nvidia-nvtx==13.3.29
  CPATH="$PWD/.cuda-headers/nvidia/cu13/include${CPATH:+:$CPATH}" ./maya.sh --setup
  

  Select wheel versions that match your toolkit. This include layout was checked with CUDA 13.3, nvidia-cuda-profiler-api==13.3.27 and nvidia-nvtx==13.3.29; other CUDA major versions can use a different layout. Use the same CPATH when rebuilding after an update. No shared toolkit files need changing.

• "unsupported GNU version": ./maya.sh --setup --host-compiler g++-12 (install g++-12 first).

• Slow, with the SSD busy all the time: not enough free RAM for the experts. Close programs or add RAM.

• Windows: answers suddenly several times slower: the card is too full and Windows moves GPU memory to RAM. Leave more of it free: START-MAYA.bat --setup --vram-reserve-mib 2048 (the expert pool's size).

• The download stopped: run ./maya.sh again; it continues where it stopped.

• Port 8080 is in use: Maya is already running, or --port 8081 starts another one.

Support

Project Maya is free and open source. If it's useful to you, you can support it at buymeacoffee.com/peasantsmith. Thank you! Speed reports, bug reports and pull requests are welcome: CONTRIBUTING.md says what helps most.

Credits and license

• People: Project Maya is created and led by Peasant Smith (@mw00, X), who owns the project, its name and the Maya models. @needmorevram (X) is a co-maintainer and contributed major features: DFlash2 drafting, EXL3 support, KV streaming, conversation checkpoints, the layer-split search, the expert profile and the CPU expert kernels. Thanks also to everyone who sent code and speed reports (contributors).
• Built on Strata (MIT License, Copyright (c) 2026 Niko1221 and the Strata contributors): Maya's engine started from Strata's and was rewritten for GLM-5.3-Flash (the expert tiers across VRAM, RAM and SSD, the layer split, MTP decoding), and its server and dashboard grew from Strata's. And on ggml / llama.cpp (MIT License, Copyright (c) 2023-2026 The ggml authors): the quantization formats, the CPU dot products and the prompt path's MMQ kernels, built from a pinned commit (third_party/ggml/LICENSE).
• EXL3 support follows exllamav3's format, codebooks, CPU expert kernel and tensor-core decoding of a tile (MIT License, Copyright (c) 2025 Turboderp; third_party/exllamav3/LICENSE). The DFlash2 drafter's forward follows beellama.cpp's (MIT License, Copyright (c) 2023-2026 The ggml authors, Anbeeld; third_party/beellama.cpp/LICENSE).
• The model: GLM-5.3-Flash by zai-org (MIT); Maya's quants and the image encoder file are made from zai-org's released weights and keep its license. The optional DFlash2 drafter (GLM-5.3-Flash-DFlash2-GGUF by Anbeeld) is CC BY-NC-ND 4.0, non-commercial use only, and is downloaded only when you choose it.
• The dashboard's font: Outfit (SIL Open Font License 1.1, serve/web/fonts/OFL.txt).
• Maya is open source under the MIT License; the notices of Strata, ggml, exllamav3 and beellama.cpp stay with every copy.