⚡ Ultra Flash

Scaling Real-Time Streaming Video Generation to High Resolutions

1K video at ~30 FPS  ·  2K video at ~18 FPS  ·  single GPU

Ultra Flash is the first framework to achieve real-time high-resolution streaming video generation, producing 1K video at ~30 FPS and 2K video at ~18 FPS on a single GPU. It cascades three key components after a low-resolution streaming generator:

1. 🎯 Architecture-Preserving T2V-to-TV2V SR Training with AIGC-oriented degradation
2. 🪶 Causal Streaming Latent Upsampler (~2M params, <5% overhead)
3. 🚄 Cascaded Streaming Optimization (sparse distillation + DPO + dynamic cache management)

📰 News

• Aug 2026 — Inference code and checkpoints released on GitHub and Hugging Face. 🎉

📋 TODO

• ✅ Release inference code and checkpoints (short + long video)
• ⬜ Release training code and configs

🔧 Requirements

• NVIDIA GPU with 24+ GB memory (RTX 4090, H200, B200 tested)
• Linux operating system
• 64 GB RAM
• Python 3.10+

🚀 Getting Started

1️⃣ Installation

conda create -n ultraflash python=3.10 -y
conda activate ultraflash
cd inference
pip install -r requirements.txt
pip install flash-attn --no-build-isolation

# Block Sparse Attention (CUDA kernel, required for SR DiT)
git clone https://github.com/mit-han-lab/Block-Sparse-Attention.git
cd Block-Sparse-Attention
pip install -e .
cd ..

python setup.py develop

2️⃣ Download Checkpoints

# Wan2.1-T2V-1.3B base model
huggingface-cli download Wan-AI/Wan2.1-T2V-1.3B --local-dir inference/wan_models/Wan2.1-T2V-1.3B

# Ultra Flash checkpoints (short video)
huggingface-cli download xin1u/UltraFlash --local-dir inference/checkpoints

# Ultra Flash checkpoints (long video, optional)
huggingface-cli download xin1u/UltraFlash-Long --local-dir inference_long/checkpoints

# Symlink shared checkpoints to avoid duplication (optional)
cd inference_long/checkpoints
ln -s ../../inference/checkpoints/sr_dit_sparse.pth .
ln -s ../../inference/checkpoints/latent_upsampler.pth .
ln -s ../../inference/checkpoints/tiny_decoder.pth .
ln -s ../../inference/checkpoints/ultra_decoder_v3.pth .
ln -s ../../inference/wan_models wan_models

The inference/checkpoints/ folder should contain:

3️⃣ Quick Start

One-click inference (2K or 1K):

cd inference
bash inference.sh

Custom inference:

cd inference
python inference.py \
    --config_path configs/self_forcing_dmd_4step.yaml \
    --checkpoint_path checkpoints/self_forcing_dmd.pt \
    --data_path prompts/examples.txt \
    --output_folder outputs/ \
    --use_ema \
    --tiny_decoder \
    --torch_compile \
    --compile_sr_dit

LR-only mode (480P, for comparison):

cd inference
python inference.py \
    --config_path configs/self_forcing_dmd_4step.yaml \
    --checkpoint_path checkpoints/self_forcing_dmd.pt \
    --data_path prompts/examples.txt \
    --output_folder outputs_lr/ \
    --use_ema \
    --lr_only

🎬 Long Video Generation

The inference_long/ directory supports long video generation (~10 seconds, 120 latent frames) using LongLive with LoRA adapters and the same SR cascade pipeline.

Additional checkpoints for long video

The inference_long/checkpoints/ folder requires:

Run long video inference

cd inference_long
python inference_sr.py \
    --config_path configs/longlive_inference_sr.yaml \
    --use_sparse_sr \
    --tiny_decoder \
    --torch_compile \
    --compile_sr_dit

⚙️ Key Arguments

Short video (inference/inference.py)

Long video (inference_long/inference_sr.py)

Inherits most arguments from the YAML config. Key CLI overrides:

🧠 Dynamic Cache Management (DCM)

Ultra Flash introduces three inference-time optimizations:

🏗️ Architecture

Text Prompt
    │
    ▼
┌─────────────────────────┐
│  Self-Forcing Generator  │  480P, 4 denoising steps, streaming
│  (Wan2.1-1.3B causal)    │
└───────────┬─────────────┘
            │ LR latents (16×60×104)
            ▼
┌─────────────────────────┐
│  Causal Latent Upsampler │  2× or 3× spatial upsampling in latent space
│  (~2M params, Conv2D)    │  Causal memory for temporal coherence
└───────────┬─────────────┘
            │ HR latents (16×120×208)
            ▼
┌─────────────────────────┐
│  Sparse SR DiT           │  Single-step denoising, block-sparse attention
│  (1.3B, causal)          │  Adaptive top-k with local window
└───────────┬─────────────┘
            │ Refined HR latents
            ▼
┌─────────────────────────┐
│  Tiny Decoder (HR)       │  Latent → Pixel (960×1664 or 1440×2496)
│  (Causal Memory Network) │  Sequential streaming mode
└───────────┬─────────────┘
            │
            ▼
       2K Video Output (~18 FPS)

🏋️ Training

Training code is coming soon. See the train/ directory for updates.

🤗 Acknowledgements

This work builds upon the following excellent open-source projects:

• FlashVSR — Block-sparse attention for streaming video super-resolution
• Self-Forcing — Real-time autoregressive video generation with self-rollout training
• TAEHV — Tiny Autoencoder for high-resolution video decoding
• Wan2.1 — Video DiT foundation model

We thank the authors for their outstanding contributions to the community.

📞 Contact

Please feel free to contact us if there is any question (luxion@mail.ustc.edu.cn).

📜 Citation

@inproceedings{luxury2026ultraflash,
  title={Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions},
  author={Huang, Jie and Fan, Zihao and Ma, Xiaoxiao and Li, Yuming and Zhuang, Jun-hao and Xue, Zeyue and Fu, Siming and Li, Haoran and Zhong, Mingchen and Zhang, Guohui and Ma, Shichen and Liu, Yijun and Shi, Jiaqi and Ma, Yanwen and Su, Yaofeng and Wang, Haoyu and Li, Yaowei and Zhang, Songchun and Jin, Weiyang and Bian, Yuxuan and Zhang, Shiyi and Xu, Haojun and Lu, Shuai and Han, Xin and Tang, Wei and Huang, Haoyang and Duan, Nan},
  booktitle={arXiv preprint},
  year={2026}
}

📄 License

This project is released under the Apache 2.0 License.