GA3C: Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU A hybrid CPU/GPU version of the Asynchronous Advantage Actor-Critic (A3C) algorithm, currently the state-of-the-art method in reinforcement learning for various gaming tasks. This CPU/GPU implementation, based on TensorFlow, achieves a significant speed up compared to a similar CPU implementation. The script cleans the checkpoints folder, which contains the network models saved during the training process, as well as removing , which is a log
of the scores achieved during training. Remember to save your trained models and scores in a different folder if needed before cleaning. launches the training procedure, following the parameters in . You can modify the training parameters directly in , or pass them as argument to . , launching overwrites the starting value of the learning rate in with the one passed as argument (see below). You may want to modify for your particular needs. 5000 RPPS: 822] [PPS: 823 TPS: 183] [NT: 2
] . **PPS** (predictions per second) demonstrates the speed of processing frames, while **Score** shows the achieved score. **RPPS** and **RScore** are the rolling average of the above values. To stop the training procedure, adjuts in propoerly, or simply use ctrl + c. ### If you want to continue training a model, set in , and set to the episode number you want to load. Be sure that the corresponding model has been saved in the checkpoints folder (the model name includes the number of
the episode). Be sure not to use if you want to stop and then continue training! How to play a game with a trained agent? ### Run You may want to modify this script for your particular needs. How to change the game, configurations, etc. ### All the configurations are in As mentioned before, one useful way of modifying a config is to pass it as an argument to . For example, to save the models while training, just run: . Sample learning curves
Typical learning curves for Pong and Boxing are shown here. These are easily obtained from the results. References ### If you use this code, please refer to our ICLR 2017 paper: This work was first presented in an oral talk at the The 1st International Workshop on Efficient Methods for Deep Neural Networks, NIPS Workshop, Barcelona (Spain), Dec.
Frequently asked questions
How many stars and forks are recorded?
NVlabs/GA3C has 661 stars and 193 forks.
What is the repository rank?
NVlabs/GA3C is ranked 15,340.
What language and category are recorded?
NVlabs/GA3C has 661 stars, uses Python and belongs to Model Development.
Which repository is closest in rank?
NVlabs/GA3C is compared with TIGER-AI-Lab/AnyV2V, ranked 15,341 with 661 stars.
README
GA3C: Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU
A hybrid CPU/GPU version of the Asynchronous Advantage Actor-Critic (A3C) algorithm, currently the state-of-the-art method in reinforcement learning for various gaming tasks. This CPU/GPU implementation, based on TensorFlow, achieves a significant speed up compared to a similar CPU implementation.
Run sh _clean.sh first, and then sh _train.sh.
The script _clean.sh cleans the checkpoints folder, which contains the network models saved during the training process, as well as removing results.txt, which is a log of the scores achieved during training.
Remember to save your trained models and scores in a different folder if needed before cleaning.
_train.sh launches the training procedure, following the parameters in Config.py.
You can modify the training parameters directly in Config.py, or pass them as argument to _train.sh.
E.g., launching sh _train.sh LEARNING_RATE_START=0.001 overwrites the starting value of the learning rate in Config.py with the one passed as argument (see below).
You may want to modify _train.sh for your particular needs.
PPS (predictions per second) demonstrates the speed of processing frames, while Score shows the achieved score. RPPS and RScore are the rolling average of the above values.
To stop the training procedure, adjuts EPISODES in Config.py propoerly, or simply use ctrl + c.
How to continue training a model?
If you want to continue training a model, set LOAD_CHECKPOINTS=True in Config.py, and set LOAD_EPISODE to the episode number you want to load.
Be sure that the corresponding model has been saved in the checkpoints folder (the model name includes the number of the episode).
Be sure not to use _clean.sh if you want to stop and then continue training!
How to play a game with a trained agent?
Run _play.sh
You may want to modify this script for your particular needs.
How to change the game, configurations, etc.?
All the configurations are in Config.py
As mentioned before, one useful way of modifying a config is to pass it as an argument to _train.sh. For example, to save the models while training, just run: train.sh TRAINERS=4.
Sample learning curves
Typical learning curves for Pong and Boxing are shown here. These are easily obtained from the results.txt file.
@conference{babaeizadeh2017ga3c,
title={Reinforcement Learning thorugh Asynchronous Advantage Actor-Critic on a GPU},
author={Babaeizadeh, Mohammad and Frosio, Iuri and Tyree, Stephen and Clemons, Jason and Kautz, Jan},
booktitle={ICLR},
biurl={https://openreview.net/forum?id=r1VGvBcxl},
year={2017}
}
@article{babaeizadeh2016ga3c,
title={{GA3C:} {GPU}-based {A3C} for Deep Reinforcement Learning},
author={Babaeizadeh, Mohammad and Frosio, Iuri and Tyree, Stephen and Clemons, Jason and Kautz, Jan},
journal={NIPS Workshop},
biurl={arXiv preprint arXiv:1611.06256},
year={2016}
}
