← Open Source
reinforcement-learning-kr

lets-do-irl

Inverse RL algorithms (APP, MaxEnt, GAIL, VAIL).

Model DevelopmentReinforcement LearningPython
Open on GitHub
Momentum
+0stars in 24 hours0.0%
779
Stars
113
Forks
+0
This week
4
Contributors
Created 2018-12-09 · Updated 2026-08-31 · #13909 today
Top developers

reinforcement-learning-kr/lets-do-irl is ranked 13,909 with 779 stars.

Same-category comparison

reinforcement-learning-kr/lets-do-irl is ranked 13,909 among these 8 peers; mit-han-lab/anycost-gan is ranked 13,906.

Same-category comparison
NameRankMetrics
scverse/anndata13,901
Language:
Python
Stars:
780
Forks:
220
Description:
Annotated data.
bayesflow-org/bayesflow13,903
Language:
Python
Stars:
779
Forks:
92
Description:
A Python library for simulation-based inference with deep learning.
ethanfetaya/NRI13,904
Language:
Python
Stars:
779
Forks:
167
Description:
Neural relational inference for interacting systems - pytorch.
mit-han-lab/anycost-gan13,906
Language:
Python
Stars:
779
Forks:
98
Description:
[CVPR 2021] Anycost GANs for Interactive Image Synthesis and Editing.
ZhaoZhibin/DL-based-Intelligent-Diagnosis-Benchmark13,915
Language:
Python
Stars:
778
Forks:
186
Description:
Source codes for the paper "Deep Learning Algorithms for Rotating Machinery Intelligent Diagnosis: An Open Source Benchmark Study".
bnu-wangxun/Deep_Metric13,918
Language:
Python
Stars:
778
Forks:
144
Description:
Deep Metric Learning.
ikergarcia1996/Self-Driving-Car-in-Video-Games13,924
Language:
Python
Stars:
778
Forks:
114
Description:
A deep neural network that learns to drive in video games.
unitreerobotics/unitree_lerobot13,926
Language:
Python
Stars:
778
Forks:
137
Description:
The unitree_il_lerobot open-source project is a modification of the LeRobot open-source training framework, enabling the training and testing of data collected using the dual-arm dexterous hands of Unitree's G1 robot.
Key parameters and specifications
Key parameters and specifications
ParameterValue
LanguagePython
CategoryModel Development
SubcategoryReinforcement Learning
KeywordsRL
Stars779
Forks113
Contributors4
Created date2018-12-09
Updated date2026-08-31
Stars in 1 day0
Stars in 7 days0
Rank13,909
Previous rank13,793
Rank change-116
7-day stars rank14,180
Previous 7-day stars rank14,199
7-day stars rank change19
Total stars rank13,639
Previous total stars rank13,623
Total stars rank change-16
CountrySouth Korea
README summary

Let's do Inverse RL Introduction This repository contains PyTorch (v0. 1) implementations of **Inverse Reinforcement Learning (IRL)** algorithms. Apprenticeship Learning via Inverse Reinforcement Learning [2] Maximum Entropy Inverse Reinforcement Learning [4] Generative Adversarial Imitation Learning [5] Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow [6] We have implemented and trained the agents with the IRL algorithms using the following environments. OpenAI GYM Mountain car Mujoco Hopper For reference, reviews of below papers related to

IRL (in Korean) are located in Let's do Inverse RL Guide. , "Algorithms for Inverse Reinforcement Learning", ICML 2000. , "Apprenticeship Learning via Inverse Reinforcement Learning", ICML 2004. , "Maximum Entropy Inverse Reinforcement Learning", AAAI 2008. , "Generative Adversarial Imitation Learning", NIPS 2016. , "Variational Discriminator Bottleneck. Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow", ICLR 2019. Table of Contents Let's do Inverse RL Introduction Table of Contents Mountain car Mujoco Hopper Reference Implementation team members Mountain car We have implemented , using

Q-learning as RL step in environment. Expert's demonstrations Navigate to in lets-do-irl/mountaincar/app/expert_demo or lets-do-irl/mountaincar/maxent/expert_demo. Shape of expert's demonstrations is (20, 130, 3); (number of demonstrations, length of demonstrations, states and actions of demonstrations) If you make demonstrations, Navigate to in lets-do-irl/mountaincar/app/expert_demo or lets-do-irl/mountaincar/maxent/expert_demo. Train & Test APP Navigate to lets-do-irl/mountaincar/app folder. **Train

** the agent wtih without rendering. If you want to test , **Test** the agent with the saved model in folder. MaxEnt Navigate to lets-do-irl/mountaincar/maxent folder. **Train** the agent wtih without rendering. If you want to test , **Test** the agent with the saved model in folder. Trained Agent We have trained the agents with two different IRL algortihms using environment. Installation Mac OS (in Korean) Ubuntu 2. Expert's demonstrations Navigate to in lets-do-irl/mujoco/gail/expert

_demo or lets-do-irl/mujoco/vail/expert_demo. Shape of expert's demonstrations is (50000, 14); (number of demonstrations, states and actions of demonstrations) We used demonstrations that get scores between about 2200 and 2600 on average. If you want to make demonstrations, Navigate to in lets-do-irl/mojoco/ppo folder. Also, you can see detailed implementation story (in Korean) of PPO in PG Travel implementation story. Train & Test GAIL Navigate to lets-do-irl/mujoco/gail folder. **Train

** the agent wtih without rendering. If you want to **Continue training** from the saved checkpoint, Note that file should be in the folder. If you want to test , **Test** the agent with the saved model in the folder. VAIL Navigate to lets-do-irl/mujoco/vail folder. **Train** the agent wtih without rendering. If you want to **Continue training** from the saved checkpoint, Note that file should be in the folder. If you want to test , **Test** the agent with

the saved model in the folder. Tensorboard Note that the results of trainings are automatically saved in folder. TensorboardX is

Frequently asked questions

How many stars and forks are recorded?

reinforcement-learning-kr/lets-do-irl has 779 stars and 113 forks.

What is the repository rank?

reinforcement-learning-kr/lets-do-irl is ranked 13,909.

What language and category are recorded?

reinforcement-learning-kr/lets-do-irl has 779 stars, uses Python and belongs to Model Development.

Which repository is closest in rank?

reinforcement-learning-kr/lets-do-irl is compared with mit-han-lab/anycost-gan, ranked 13,906 with 779 stars.

README

Let's do Inverse RL

Introduction

This repository contains PyTorch (v0.4.1) implementations of Inverse Reinforcement Learning (IRL) algorithms.

  • Apprenticeship Learning via Inverse Reinforcement Learning [2]
  • Maximum Entropy Inverse Reinforcement Learning [4]
  • Generative Adversarial Imitation Learning [5]
  • Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow [6]

We have implemented and trained the agents with the IRL algorithms using the following environments.

For reference, reviews of below papers related to IRL (in Korean) are located in Let's do Inverse RL Guide.

Table of Contents

Mountain car

We have implemented APP, MaxEnt using Q-learning as RL step in MountainCar-v0 environment.

1. Information

2. Expert's demonstrations

Navigate to expert_demo.npy in lets-do-irl/mountaincar/app/expert_demo or lets-do-irl/mountaincar/maxent/expert_demo.

Shape of expert's demonstrations is (20, 130, 3); (number of demonstrations, length of demonstrations, states and actions of demonstrations)

If you make demonstrations, Navigate to make_expert.py in lets-do-irl/mountaincar/app/expert_demo or lets-do-irl/mountaincar/maxent/expert_demo.

3. Train & Test

APP

Navigate to lets-do-irl/mountaincar/app folder.

Train the agent wtih APP without rendering.

python train.py

If you want to test APP, Test the agent with the saved model app_q_table.npy in app/results folder.

python test.py

MaxEnt

Navigate to lets-do-irl/mountaincar/maxent folder.

Train the agent wtih MaxEnt without rendering.

python train.py

If you want to test MaxEnt, Test the agent with the saved model maxent_q_table.npy in maxent/results folder.

python test.py

4. Trained Agent

We have trained the agents with two different IRL algortihms using MountainCar-v0 environment.

Algorithms Scores / Episodes GIF
APP
MaxEnt

Mujoco Hopper

We have implemented GAIL, VAIL using PPO as RL step in Hopper-v2 environment.

1. Installation

2. Expert's demonstrations

Navigate to expert_demo.p in lets-do-irl/mujoco/gail/expert_demo or lets-do-irl/mujoco/vail/expert_demo.

Shape of expert's demonstrations is (50000, 14); (number of demonstrations, states and actions of demonstrations)

We used demonstrations that get scores between about 2200 and 2600 on average.

If you want to make demonstrations, Navigate to main.py in lets-do-irl/mojoco/ppo folder.

Also, you can see detailed implementation story (in Korean) of PPO in PG Travel implementation story.

3. Train & Test

GAIL

Navigate to lets-do-irl/mujoco/gail folder.

Train the agent wtih GAIL without rendering.

python main.py

If you want to Continue training from the saved checkpoint,

python main.py --load_model ckpt_4000_gail.pth.tar
  • Note that ckpt_4000_gail.pth.tar file should be in the mujoco/gail/save_model folder.

If you want to test GAIL, Test the agent with the saved model ckpt_4000_gail.pth.tar in the mujoco/gail/save_model folder.

python test.py --load_model ckpt_4000_gail.pth.tar
  • Note that ckpt_4000_gail.pth.tar file should be in the mujoco/gail/save_model folder.

VAIL

Navigate to lets-do-irl/mujoco/vail folder.

Train the agent wtih VAIL without rendering.

python main.py

If you want to Continue training from the saved checkpoint,

python main.py --load_model ckpt_4000_vail.pth.tar
  • Note that ckpt_4000_vail.pth.tar file should be in the mujoco/vail/save_model folder.

If you want to test VAIL, Test the agent with the saved model ckpt_4000_vail.pth.tar in the mujoco/vail/save_model folder.

python test.py --load_model ckpt_4000_vail.pth.tar
  • Note that ckpt_4000_vail.pth.tar file should be in the mujoco/vail/save_model folder.

4. Tensorboard

Note that the results of trainings are automatically saved in logs folder. TensorboardX is the Tensorboard-like visualization tool for Pytorch.

Navigate to the lets-do-irl/mujoco/gail or lets-do-irl/mujoco/vail folder.

tensorboard --logdir logs

5. Trained Agent

We have trained the agents with two different IRL algortihms using Hopper-v2 environment.

Algorithms Scores / Iterations (total sample size : 2048)
PPO (to compare)
GAIL
VAIL
Total

Reference

We referenced the codes from below repositories.

Implementation team members

Dongmin Lee (project manager) : Github, Facebook

Seungje Yoon : Github, Facebook

Seunghyun Lee : Github, Facebook

Geonhee Lee : Github, Facebook

Nearby in the ranking
Similar items