PyBullet Gymnasium environments for single and multi-agent reinforcement learning of quadcopter control

[!TIP] For research work with symbolic dynamics and constraints, also try safe-control-gym

For GPU-accelerated, differentiable, JAX-based simulation, also try crazyflow

For real-world deployment of PX4/ArduPilot + ROS2 + JetPack, use aerial-autonomy-stack

gym-pybullet-drones

This is a minimalist refactoring of the original gym-pybullet-drones repository, designed for compatibility with gymnasium, stable-baselines3 2.0, and betaflight SITL.

NEWS: gym-pybullet-drones was featured in GitHub's Maintainer Spotlight 2026

NOTE: if you want to access the original IROS 2021 codebase, please git checkout [paper|master]

formation flight control info

Installation

Tested on Intel x64/Ubuntu 24.04 and Apple Silicon/macOS 26.

git clone https://github.com/learnsyslab/gym-pybullet-drones.git
cd gym-pybullet-drones/

conda create -n drones python=3.12
conda activate drones

# Beyond Python 3.10, `pybullet` has no pre-built wheel
# On Ubuntu, install `gcc` to let `pip3 install` build `pybullet`
sudo apt install build-essential
# On macOS, build and install `pybullet` with
CFLAGS="-Dfdopen=fdopen" pip install pybullet --no-cache-dir

pip3 install -e .

# check installed packages with `conda list`, deactivate with `conda deactivate`, remove with `conda remove -n drones --all`

Use

Control examples

cd gym_pybullet_drones/examples/
python3 pid.py
python3 pid_velocity.py
python3 mrac.py

Downwash effect example

cd gym_pybullet_drones/examples/
python3 downwash.py

Reinforcement learning examples (SB3's PPO)

cd gym_pybullet_drones/examples/

# single agent, task: single drone hover at z == 1.0
python learn.py
LATEST_MODEL=$(ls -t results | head -n 1) && python play.py --model_path "results/${LATEST_MODEL}/best_model.zip"

# multi-agent, task: 2-drone hover at z == 1.2 and 0.7
python learn.py --multiagent true
LATEST_MODEL=$(ls -t results | head -n 1) && python play.py --multiagent true --model_path "results/${LATEST_MODEL}/best_model.zip"

rl example marl example

Run all tests

# from the repo's top folder
cd gym-pybullet-drones/
pytest tests/

Betaflight SITL example (Ubuntu only)

# one-time setup: from the repo's top folder, build one SITL executable per drone (e.g. 2), if needed, `apt install curl`
cd gym-pybullet-drones/
./gym_pybullet_drones/assets/clone_bfs.sh 2

# run the example
cd gym_pybullet_drones/examples/
python3 beta.py --num_drones 2
# --num_drones must be <= the number passed to clone_bfs.sh

Citation

If you wish, please cite our IROS 2021 paper (and original codebase) as

@INPROCEEDINGS{panerati2021learning,
      title={Learning to Fly---a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control}, 
      author={Jacopo Panerati and Hehui Zheng and SiQi Zhou and James Xu and Amanda Prorok and Angela P. Schoellig},
      booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
      year={2021},
      volume={},
      number={},
      pages={7512-7519},
      doi={10.1109/IROS51168.2021.9635857}
}

References

Erwin Coumans and Yunfei Bai (2023) PyBullet Quickstart Guide
Carlos Luis and Jerome Le Ny (2016) Design of a Trajectory Tracking Controller for a Nanoquadcopter
Nathan Michael, Daniel Mellinger, Quentin Lindsey, Vijay Kumar (2010) The GRASP Multiple Micro-UAV Testbed
Benoit Landry (2014) Planning and Control for Quadrotor Flight through Cluttered Environments
Julian Forster (2015) System Identification of the Crazyflie 2.0 Nano Quadrocopter
Antonin Raffin, Ashley Hill, Maximilian Ernestus, Adam Gleave, Anssi Kanervisto, and Noah Dormann (2019) Stable Baselines3
Guanya Shi, Xichen Shi, Michael O’Connell, Rose Yu, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, and Soon-Jo Chung (2019) Neural Lander: Stable Drone Landing Control Using Learned Dynamics
C. Karen Liu and Dan Negrut (2020) The Role of Physics-Based Simulators in Robotics
Yunlong Song, Selim Naji, Elia Kaufmann, Antonio Loquercio, and Davide Scaramuzza (2020) Flightmare: A Flexible Quadrotor Simulator

UTIAS / Learning Systems and Robotics Lab / Vector Institute / University of Cambridge's Prorok Lab