PixelDiT

PixelDiT & PixelDiT2

Pixel-space diffusion transformers for image generation

NVIDIA · University of Rochester

Papers and Resources

Work	Publication	Resources
PixelDiT2: Representation-Grounded Pixel Diffusion Transformers	NeurIPS 2026
PixelDiT: Pixel Diffusion Transformers for Image Generation	CVPR 2026 Oral

Authors

PixelDiT2

Yongsheng Yu · Wei Xiong · Yichen Sheng · Shiqiu Liu · Jiebo Luo

PixelDiT

Yongsheng Yu · Wei Xiong · Weili Nie · Yichen Sheng · Shiqiu Liu · Jiebo Luo

Affiliation: NVIDIA · University of Rochester

Project lead and main advisor: Wei Xiong

News

• 2026/09 — PixelDiT2 is accepted to NeurIPS 2026, a representation-grounded pixel diffusion model that converges much faster.
• 2026/06 — PixelDiT is a CVPR 2026 Best Paper Finalist.
• 2026/06 — Added post-modulation for PiT blocks to improve training stability.
• 2026/04 — PixelDiT training and inference code, and pretrained models, are released.
• 2026/02 — PixelDiT is accepted to CVPR 2026 Oral.
• 2025/11 — The PixelDiT paper is released.

Introduction

PixelDiT is a series of work on pixel-space generative modeling, aiming to remove the need of VAEs which are widely used in traditional latent generative models and perform geneartion directly in pixel space.

• PixelDiT: Our first pixel-space diffusion models. We release both the models on ImageNet and a 1.3B Text-to-Image model that can generate 1024x1024 resolution images.
• PixelDiT2: We have a deeper investigation on how representation learning can guide the content generation for pixel diffusion. We propose to use representations to ground the generation process. It converges significantly faster and obtains better results on ImageNet. Please read the details in our paper.

Models and results

Note: Our models are resumed every 4 hours, using the timestamp as the random seed each time. As a result, the final training outcome may have a slight gap compared to a continuous run without intermediate resumes.

Class-to-image · ImageNet

Results use 50,000 samples. PixelDiT2 uses Heun with 50 steps; PixelDiT uses FlowDPMSolver with 100 steps. The class-to-image guide lists all checkpoints and the corresponding sampling commands.

Text-to-image · PixelDiT-T2I

See the text-to-image guide for training and inference.

ComfyUI. PixelDiT-T2I is also available in ComfyUI. The Comfy-Org/PixelDiT repository provides repackaged weights (bf16 and mxfp8) and a ready-to-use text-to-image workflow. The repository also hosts PiD.

Quick start

Install the requirements for the model family you want to use, in separate Python environments:

PixelDiT2 uses Python 3.10 and pinned PyTorch/CUDA dependencies. The PixelDiT file retains the original dependency specification; requirements.txt remains its compatibility entrypoint. C2I evaluation also differs: PixelDiT2 uses the LTH14 torch-fidelity fork with JiT statistics, while PixelDiT uses ADM in a separate evaluation environment. See the guides for details.

For class-to-image generation, both model families use c2i/main.py. Choose a matching checkpoint and configuration in the model list, then follow the shared inference or training workflow.

Use the model definitions in another project

The model code in pixdit_core can be installed with pip:

python -m pip install "git+https://github.com/NVlabs/PixelDiT"
# PixelDiT2's grounding encoder also needs timm:
python -m pip install "pixeldit[pixeldit2] @ git+https://github.com/NVlabs/PixelDiT"

Append @ to the URL to pin a version. The package is named pixeldit and is imported as pixdit_core:

from pixdit_core.pixeldit_t2i import PixDiT_T2I    # PixelDiT text-to-image
from pixdit_core.pixeldit_c2i import PixDiT        # PixelDiT class-to-image
from pixdit_core.pixeldit2_c2i import PixelDiT2    # PixelDiT2 class-to-image

Repository Structure

├── pixdit_core/      # Shared model definitions
│   ├── pixeldit2_c2i.py   # PixelDiT2  (single-path patch DiT + grounding)
│   ├── grounding.py       # PixelDiT2  frozen DINOv3 encoder and projection P_g
│   ├── pixeldit_c2i.py    # PixelDiT   (dual-level: patch DiT + pixel PiT)
│   └── pixeldit_t2i.py    # PixelDiT   text-to-image
├── tools/            # Checkpoint download, evaluation and GFLOPs computation
├── c2i/              # Class-to-image (PixelDiT and PixelDiT2)
└── t2i/              # Text-to-image

Compute model GFLOPs

Compute GFLOPs

Measure single-forward-pass GFLOPs for any model in this repository (run from project root). For PixelDiT2 the count includes the frozen grounding encoder.

# PixelDiT2 (ImageNet 256x256 and 512x512)
python tools/compute_flops.py --config c2i/configs/pixeldit2_h16_in256.yaml
python tools/compute_flops.py --config c2i/configs/pixeldit2_h16_in512.yaml --height 512 --width 512

# PixelDiT C2I (ImageNet 256x256, default resolution)
python tools/compute_flops.py --config c2i/configs/pix256_xl.yaml

# PixelDiT T2I at 1024x1024
python tools/compute_flops.py --config t2i/configs/PixelDiT_1024px_pixel_diffusion_stage3.yaml --height 1024 --width 1024

Acknowledgements

We would like to thank the authors of PixNerd and SANA for sharing their code. We also thank the SANA team for sharing their text-to-image training data.

Citation

If you find this work useful, please cite:

@inproceedings{yu2026pixeldit2,
      title={PixelDiT2: Representation-Grounded Pixel Diffusion Transformers},
      author={Yongsheng Yu and Wei Xiong and Yichen Sheng and Shiqiu Liu and Jiebo Luo},
      booktitle={Conference on Neural Information Processing Systems (NeurIPS)},
      year={2026},
}

@inproceedings{yu2026pixeldit,
      title={PixelDiT: Pixel Diffusion Transformers for Image Generation},
      author={Yongsheng Yu and Wei Xiong and Weili Nie and Yichen Sheng and Shiqiu Liu and Jiebo Luo},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2026},
}