Towards the Era of Post-Training for Autonomous Driving

The missing infrastructure for Physical AI post-training in AD. Open-source. Production-validated.

Table of Contents

• Highlights
• News
• Benchmark
  • Qualitative Results — Closed-Loop Simulation on nuPlan
  • On-Road Deployment — Night Urban Driving
• System Architecture
• Roadmap
• Getting Started
  • Documentation Overview
  • Installation
  • Quick Test
  • Deep Dive by Module
    • SimEngine - Photorealistic Closed-Loop Simulation
    • AlgEngine - End-to-End Model Training & Fine-Tuning
    • Scene Reconstruction - 3D Gaussian Splatting-based method, MTGS
• Citation
• Contributing
• License
• Acknowledgements
• Related Resources

Highlights

• WorldEngine is a post-training framework for Physical AI that systematically addresses the long-tail safety-critical data scarcity problem in autonomous driving.
• Data-driven long-tail discovery: Failure-prone scenarios are automatically identified from real-world driving logs by the pre-trained agent itself — no manual design, no synthetic perturbations.
• Photorealistic interactive simulation via 3D Gaussian Splatting (3DGS): Each discovered scenario is reconstructed into a fully controllable, real-time-renderable simulation environment with independent dynamic agent manipulation.
• Behavior-driven scenario generation: Leverages Behavior World Model (BWM) to generalize and synthesize diverse traffic variations from existing long-tail scenarios, expanding sparse safety-critical events into a dense, learnable distribution.
• RL-based post-training on synthesized safety-critical rollouts substantially outperforms scaling pre-training data alone — competitive with a ~10× increase in pre-training data.
• Production-scale validation: Deployed on a mass-produced ADAS platform trained on 80,000+ hours of real-world driving logs, reducing simulated collision rate by up to 45.5% and achieving zero disengagements in a 200 km on-road test.

News

• [2026/09/15] Stable checkpoints released.
• [2026/06/19] Paper released on arXiv. See World Engine: Towards the Era of Post-Training for Autonomous Driving.
• [2026/04/09] Official dataset released. See OpenDriveLab/WorldEngine or OpenDriveLab/WorldEngine (ModelScope)
• [2026/04/10] Official code repository established.

Benchmark

We compare different post-training paradigms on the nuPlan dataset, evaluating on both open-loop and closed-loop metrics across common and rare driving scenarios.

Metric notes:

• Open-loop PDMS is aligned with NAVSIM v1.1 PDM Score. Common denotes the standard navtest split; Rare denotes the navtest_failures subset — failure-prone rare-case scenarios extracted from navtest.
• Closed-loop Success Rate (SR) is computed as NC × DAC (no-at-fault-collision score × drivable-area compliance score), reported as a percentage.
• Closed-loop Ego Progress (EP) measures the route progress made by the ego vehicle during SimEngine closed-loop testing, reflecting whether the agent makes meaningful forward progress rather than merely avoiding collision or off-road failure.
• Closed-loop PDMS* is the PDM Score obtained via SimEngine closed-loop testing, where the planner interacts with reactive agents in simulation under real-time rendering.

Training notes:

• Rare logs are failure-prone scenarios automatically extracted from navtrain by the pre-trained agent itself (see Rare Case Extraction).
• Common logs are the standard cases in navtrain.

VADv2 (Base Planner)

Key findings:

• Post-training on rare logs substantially improves rare open-loop PDMS over supervised fine-tuning (59.20 vs. 52.55), but does not improve rare closed-loop SR, indicating that fixed rare logs alone are insufficient for robust interactive behaviour.
• Post-training on common logs provides limited long-tail benefit and degrades rare closed-loop performance, reducing SR from 73.66% to 69.63% and PDMS$^\ast$ from 60.98 to 60.21, confirming the importance of long-tail event discovery.
• The full WorldEngine pipeline achieves the best overall rare closed-loop performance, with the highest SR (88.89%) and PDMS$^\ast$ (70.12). It improves rare closed-loop SR by +15.23 percentage points and PDMS$^\ast$ by +9.14 over the base model, while maintaining strong common open-loop performance.

HydraMDP

The base model and WorldEngine rows match Table S1 of the paper. Additional ablations follow the selected HydraMDP experiment records: pure IL fine-tuning for the supervised row, and reward shaping enabled, RL fine-tuning enabled, PG = 0.01, entropy = 0 for the RL rows. Closed-loop values use the reactive evaluation results.

WorldEngine improves HydraMDP's rare closed-loop SR by +6.48 percentage points, EP by +4.97, and PDMS$^\ast$ by +5.80, matching the gains reported in Table S1.

Qualitative Results — Closed-Loop Simulation on nuPlan

Each pair shows the Base model vs WorldEngine post-trained model on the same rare-case scenario. Left: front-camera rendering; Right: BEV trajectory visualization.

On-Road Deployment — Night Urban Driving

Zero disengagements in 200 km on-road testing on a mass-produced ADAS platform.

System Architecture

WorldEngine consists of two tightly coupled subsystems:

Roadmap

• [x] Core platform integration (SimEngine + AlgEngine)
• [x] Multi-GPU distributed simulation and training
• [x] Rare case extraction and fine-tuning pipeline
• [x] Comprehensive documentation and usage guides
• [x] Hugging Face / ModelScope dataset
• [x] Open-source release (code, data, early pre-trained models)
• [x] arXiv preprint
• [x] Stable pre-trained models
• [ ] Behavior World Model integration

Getting Started

Documentation Overview

WorldEngine provides comprehensive guides for each stage of your workflow:

Installation

WorldEngine requires two separate conda environments due to different Python requirements.

Full installation guide: docs/installation.md

Quick Test

Verify your installation with a pre-trained model:

# Set up environment variable
export WORLDENGINE_ROOT=$(pwd)

# Option 1: Single GPU test 
bash scripts/closed_loop_test.sh

# Option 2: Multi-GPU test (Default 8 GPUs)
bash scripts/multigpu_closed_loop_test.sh

What this does:

• Loads a pre-trained VADv2 model (50% training data, epoch 8)
• Runs closed-loop simulation on 288 rare-case test scenarios
• Evaluates with navsim v1 PDMS (collision avoidance, progress, comfort, etc.)
• Saves results to experiments/closed_loop_exps/e2e_vadv2_50pct/navtest_failures_NR/

Detailed quick start tutorial: docs/quick_start.md

Deep Dive by Module

After the quick test, explore each subsystem in detail:

SimEngine - Photorealistic Closed-Loop Simulation

Learn how to run simulations, generate rollouts, and test models:

• Rollout scripts for data generation (no model required)
• Testing scripts for model evaluation (single/multi-GPU)
• Ray distributed simulation for large-scale testing
• Reactive vs non-reactive agent modes
• Configuration guide for all Hydra parameters

SimEngine Usage Guide

AlgEngine - End-to-End Model Training & Fine-Tuning

Learn how to train models, extract rare cases, and fine-tune:

• Training from scratch
• Open-loop evaluation on test sets
• Rare case extraction from evaluation failures
• RL-based fine-tuning on long-tail scenarios
• Multi-GPU training with distributed data parallel

AlgEngine Usage Guide

Scene Reconstruction - 3D Gaussian Splatting-based method, MTGS

WorldEngine's simulation environments are powered by 3D Gaussian Splatting (MTGS):

• Multi-traversal reconstruction from nuPlan data
• Photorealistic rendering for closed-loop simulation
• Asset generation for SimEngine scenes

MTGS Repository

Citation

If any parts of our work help your research, please consider citing us and giving a star to our repository:

@misc{li2026worldengine,
  title={World Engine: Towards the Era of Post-Training for Autonomous Driving}, 
  author={Tianyu Li and Li Chen and Caojun Wang and Haochen Liu and Kashyap Chitta and Zhenjie Yang and Yuhang Lu and Naisheng Ye and Yihang Qiu and Yufei Wang and Luoxi Zou and Jiaxin Peng and Jin Pan and Zhaoyu Su and Andrei Bursuc and Shengbo Eben Li and Andreas Geiger and Peng Su and Hongyang Li},
  year={2026},
  eprint={2606.19836},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2606.19836}, 
}

Contributing

We welcome contributions from the community! Whether you want to:

• Report bugs - Open an Issue
• Improve documentation - Submit a Pull Request
• Contribute code - Fork, develop, and submit a PR

Please read our contributing guidelines before submitting PRs.

For questions:

1. Check the documentation first
2. Search existing Issues

License

All content in this repository is under the Apache-2.0 license.

The released data is based on nuPlan and is under the CC-BY-NC-SA 4.0 license.

Acknowledgements

WorldEngine builds on top of the following works. We sincerely thank the authors for their contributions:

• MTGS — Multi-Traversal Gaussian Splatting, the scene reconstruction backbone powering our photorealistic simulation environments.

  @article{li2025mtgs,
    title={MTGS: Multi-Traversal Gaussian Splatting},
    author={Li, Tianyu and Qiu, Yihang and Wu, Zhenhua and Lindstr{\"o}m, Carl and Su, Peng and Nie{\ss}ner, Matthias and Li, Hongyang},
    journal={arXiv preprint arXiv:2503.12552},
    year={2025}
  }
  

• Nexus (Decoupled Diffusion) — Decoupled Diffusion Sparks Adaptive Scene Generation, enabling our augmented scenario generation pipeline.

  @inproceedings{zhou2025nexus,
    title={Decoupled Diffusion Sparks Adaptive Scene Generation},
    author={Zhou, Yunsong and Ye, Naisheng and Ljungbergh, William and Li, Tianyu and Yang, Jiazhi and Yang, Zetong and Zhu, Hongzi and Petersson, Christoffer and Li, Hongyang},
    booktitle={ICCV},
    year={2025}
  }
  

• Optimization-Guided Diffusion — Optimization-Guided Diffusion for Interactive Scene Generation, underpinning behaviour-driven scenario synthesis.

  @inproceedings{li2025optimization,
    title={Optimization-Guided Diffusion for Interactive Scene Generation},
    author={Li, Shihao and Ye, Naisheng and Li, Tianyu and Chitta, Kashyap and An, Tuo and Su, Peng and Wang, Boyang and Liu, Haiou and Lv, Chen and Li, Hongyang},
    booktitle={ECCV},
    year={2026}
  }
  

• AlgEngine (Reinforced Refinement) — Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Driving, the RL-based fine-tuning backbone of AlgEngine.

  @ARTICLE{liu2026r2se,
    author={Liu, Haochen and Li, Tianyu and Yang, Haohan and Chen, Li and Wang, Caojun and Guo, Ke and Tian, Haochen and Li, Hongchen and Li, Hongyang and Lv, Chen},
    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
    title={Reinforced Refinement With Self-Aware Expansion for End-to-End Autonomous Driving},
    year={2026},
    volume={48},
    number={5},
    pages={5774-5792},
    doi={10.1109/TPAMI.2026.3653866}
  }
  

• SimScale — Learning to Drive via Real-World Simulation at Scale, providing insights into large-scale simulation-based training.

  @article{tian2025simscale,
    title={SimScale: Learning to Drive via Real-World Simulation at Scale},
    author={Haochen Tian and Tianyu Li and Haochen Liu and Jiazhi Yang and Yihang Qiu and Guang Li and Junli Wang and Yinfeng Gao and Zhang Zhang and Liang Wang and Hangjun Ye and Tieniu Tan and Long Chen and Hongyang Li},
    journal={arXiv preprint arXiv:2511.23369},
    year={2025}
  }
  

Related Resources

We acknowledge all the open-source contributors for the following projects to make this work possible:

Project	Description
	Multi-traversal Gaussian Splatting for scene reconstruction
	Large scale driving simulation
	Collaboration-friendly NeRF toolkit
	3D detection framework
	End-to-end autonomous driving framework
	Vectorized autonomous driving framework
	Non-reactive autonomous vehicle simulation benchmark
	Large-scale autonomous driving dataset
	Compositional driving simulation platform
	Distributed execution framework
	Configuration management framework

If you find WorldEngine useful, please consider giving us a star!

Quick Links: Documentation | Installation | Quick Start | Issues | Discussions

Contact: For research collaboration or questions, visit our Discussions