MAGI-2 Preview

Magi-2 Preview is a 114B-parameter unified audio-video generation model that activates just 6B parameters per token. Built on MagiMoE and co-designed across architecture, systems, and data, it explores an efficient path to scaling video generation. The architecture, the training system built around it, and the data pipeline are described in MAGI-2 Preview: Scaling Video Generation Models Efficiently; the weights are on Hugging Face at sand-ai/MAGI-2-preview.

This repository is the inference code. It generates video from a text prompt (T2V) or from a prompt plus a still image (I2V), with sound generated alongside the video and muxed into the output file. Clips are 10 seconds long, which is the only duration the model currently supports. Generation runs in two stages: magi2_preview denoises the clip at low resolution, and magi2_refiner takes that result up to 1080p.

Requirements

• NVIDIA Hopper GPUs, 8 of them.
• Python 3.12 and a recent CUDA toolkit.
• ffmpeg on PATH, to mux the audio track. Without it the video is still written, just silently.

Setup

Docker

The published image, sandai/magi-2-preview, already has the dependencies built, including the ones that need a compiler:

docker pull sandai/magi-2-preview:latest
docker run --gpus all -it -v /path/to/ckpt:/workspace/ckpt sandai/magi-2-preview:latest

There is a tag per commit as well, sandai/magi-2-preview:. Name that one when reporting a result, because latest moves; the image also records what it was built from in /etc/magi2-build-info.

Building it yourself is only necessary to change a dependency version, or to work somewhere the registry is not reachable:

docker build -t magi-2-preview:local .

From source

pip install -r requirements.txt

MAGI-2 also needs MagiAttention and MagiCompiler. The pinned revisions are recorded as build args in the Dockerfile.

Checkpoints

Neither transformer has been step-distilled, so the denoising step count is where most of the wall-clock time goes. That is the difference between the release that exists today and the one that follows:

Everything the base model loads lives in that one repository, roughly 307 GB in total. Weights are not bundled with the code; download them into ckpt/ in the repository root, which is gitignored:

pip install huggingface_hub
hf download sand-ai/MAGI-2-preview --local-dir ckpt

The directory names in that repository are the ones the configs already expect, so nothing needs renaming afterwards:

ckpt/
├── preview/                            # preview stage: 56 safetensors shards + index
├── refiner/                            # refiner stage: 3 shards + index
├── text_encoder/                       # text encoder
├── vae/                                # video VAE
│   └── Wan2.2_VAE.pth
├── stable-audio-open-1.0/              # audio VAE
└── turbo_vae/                          # fast VAE decoder
    ├── TurboV3-Wan22-TinyShallow_7_7.json
    └── checkpoint.ckpt

Each directory name below links to the folder it comes from in the Hugging Face repository:

The configs under configs/ reference these as ${MAGI2_CKPT_ROOT}/, and that variable defaults to /ckpt. To keep weights somewhere else, point it at them rather than editing the configs:

export MAGI2_CKPT_ROOT=/data/magi2-weights

Prompt enhancement

The captions the model was trained on are long and structured, so a short hand- written prompt underuses it. The pipeline can optionally rewrite the input through a prompt-enhancement (PE) step before encoding:

• text-to-video uses inference/prompt_enhancement/prompts/t2v.md
• image-to-video (when a first-frame image is provided) uses inference/prompt_enhancement/prompts/i2v.md

Both templates ask an instruction-following LLM for a structured JSON caption of the 10-second clip; the result is then rendered to readable Markdown and passed downstream. The reference client in inference/prompt_enhancement/ talks to an OpenAI-compatible endpoint — set API_KEY in enhancer.py to enable it, or subclass LLMClient for your own provider. Leave API_KEY empty to skip PE and use the raw prompt as-is.

assets/ ships example prompts that are already PE-enhanced (sample_000.txt and sample_002.txt). You can still generate without running PE — leave API_KEY empty and pass any prompt you like.

Running inference

scripts/run_demo.sh launches inference/pipeline/entry.py under torchrun on every visible GPU. It generates at 1080p, with seed 42 and the batch in assets/demo_samples.json:

bash scripts/run_demo.sh
SAMPLES=my_samples.json bash scripts/run_demo.sh   # a different batch
OUTPUT_DIR=output/run7 bash scripts/run_demo.sh

The script takes SAMPLES, OUTPUT_DIR, SEED and MASTER_PORT from the environment. Videos land in $OUTPUT_DIR/sample_000.mp4 and up, numbered by position in the batch.

A samples file is a JSON array with one entry per video. An entry carries its prompt inline as prompt or as a path in prompt_file, and a first frame in image; leaving image out makes it a T2V entry. The shipped batch runs the three stills in assets/ as I2V, the same three prompts again as T2V, and assets/sample_enhanced_t2v.json.

For a single clip, call the entry point directly:

torchrun --nproc_per_node=8 inference/pipeline/entry.py \
    --prompt "a red fox in snow" --output output/

It also takes --prompt-file, --image, --seed, --config, --output-width / --output-height, --num-inference-steps, --refiner-num-inference-steps and --deterministic. Of these only --seed, --samples and --output are reachable through run_demo.sh.

1080p runs configs/magi2_refiner.json: the preview stage generates 512x896 and the refiner takes that to 1088x1920. magi2_refiner.json extends magi2_preview.json and carries only what the refiner stage adds, so a shared setting is edited in one place.

1080p is a delivery tier, not the shape that gets generated. The VAE stride constrains every generated dimension to a multiple of 16, so the tier generates 1088 wide rather than 1080, and the video is written at that generated shape. Pass --output-width and --output-height to have the finished video rescaled to an exact size, 1080x1920 the way the reference delivers the tier.

Four environment variables decide where each large component sits between phases: MAGI2_TEXT_ENC_OFFLOAD_MODE, MAGI2_PREVIEW_OFFLOAD_MODE, MAGI2_REFINER_OFFLOAD_MODE and MAGI2_VAE_OFFLOAD_MODE, each one of cpu, gpu or roundtrip. The preview and the refiner default to roundtrip, staged in and out around the stage that needs them, because at 1080p neither fits on an 80GB card next to the other's activations.

Decoding uses the distilled turbo decoder from ckpt/turbo_vae, a temporal sliding window that runs on one rank per video. MAGI2_DETERMINISTIC=1, or --deterministic, makes the MoE scatter and the attention kernels bit-exact at some cost in speed. MAGI2_SAVE_LATENT_PATH writes the post-refiner latent of each sample to that directory.

License

Apache 2.0. See LICENSE.