AI Rank
EN
← 开源
Kim-Hammar

awesome-rl-for-cybersecurity

A curated list of resources dedicated to reinforcement learning applied to cyber security.

ListsDomain-specific
在 GitHub 打开
增长势头
+024 小时新增 Star0.0%
1.10k
Star
146
Fork
+3
本周
21
贡献者
创建于 2021-12-14 · 更新于 2026-10-15 · 今日第 11586 名
主要开发者
Kim-HammarHassanChowdhryruipcf
README

Awesome Reinforcement Learning
for Cyber Security

A curated list of resources dedicated to reinforcement learning applied to cyber security. Note that the list includes only work that uses reinforcement learning, general machine learning methods applied to cyber security are not included in this list.

For other related curated lists, see :

Table of Contents

↑ Environments

CSLE: The Cyber Security Learning Environment

     ![](imgs/csle_logo_cropped.png) 
  

  

    [CSLE: The Cyber Security Learning Environment](https://github.com/Limmen/csle)
    
      
        CSLE is a platform for evaluating and developing reinforcement learning agents for control problems in cyber security. It can be considered as a cyber range specifically designed for reinforcement learning agents. Everything from network emulation, to simulation and implementation of network commands have been co-designed to provide an environment where it is possible to train and evaluate reinforcement learning agents on practical problems in cyber security.
      
	  
	  Paper: [(2026) CSLE: A Reinforcement Learning Platform for Autonomous Security Management](https://arxiv.org/abs/2604.15590)  

	  Thesis: [(2024) Optimal Security Response to Network Intrusions in IT Systems](https://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A1912164&dswid=7946)

Continuous CyberBattleSim

    [![](imgs/c-cyberbattlesim.png)](https://github.com/terranovafr/C-CyberBattleSim)
  

  

    [C-CyberBattleSim](https://github.com/terranovafr/C-CyberBattleSim)
    
      
		  An enhanced version of Microsoft CyberBattleSim that integrates graph neural networks and language models to create generalizable, scalable continuous spaces for reinforcement learning, features an extended scenario generation pipeline utilizing Shodan and the NVD, and provides a unified framework for RL training and evaluation.  


		Paper: [(2025) Scalable and Generalizable RL Agents for Attack Path Discovery via Continuous Invariant Spaces](https://ieeexplore.ieee.org/document/11352493)
		Documentation: [Read The Docs](https://c-cyberbattlesim.readthedocs.io/en/latest/home.html)

MininetGym

     ![MininetGym](imgs/mininet-gym.png) 
  

  

    [MininetGym](https://github.com/dipi-unimore/mininet-gym)
    
      
        MininetGym is a customizable and modular SDN-based simulation framework designed for evaluating Reinforcement Learning (RL) and Multi-Agent Reinforcement Learning (MARL) strategies in cybersecurity. Based on Mininet, it offers high configurability for network traffic classification and attack detection tasks (e.g., DoS/DDoS mitigation).
      
      
        Paper: [(2026) MininetGym: A Live Demonstration of RL-Based Cybersecurity Training](https://doi.org/10.65109/VVUY3381) (AAMAS 2026)  

		          
	  	
		  Paper: [(2026) Experiences in Exploiting Reinforcement Learning for Network Traffic Classification and Attack Detection](https://doi.org/10.5220/0014300900004052) (ICAART 2026)  

		            
	  	
        Paper: [(2025) MininetGym: A modular SDN-based simulation environment for reinforcement learning in cybersecurity](https://www.sciencedirect.com/science/article/pii/S235271102500278X) (SoftwareX)   

      
	  			  
		Demo: [screensaver](https://finix77.github.io/mininetgym/screensaver.html) and [video](https://finix77.github.io/mininetgym/video.html)
      			  
		Documentation: [Read The Docs](https://finix77.github.io/mininetgym/manual.html)

CyGym

     ![](imgs/cygym.png) 
  

  

    [CyGym: A Simulation-Based Game-Theoretic Analysis Framework for Cybersecurity](https://github.com/Lan131/CyGym)

CyGym is a cybersecurity encounter simulator leveraging the OpenAI Gym framework for game-theoretic reinforcement learning research in network defense. It features realistic network topologies, a broad array of vulnerabilities and exploits including zero-day attacks, and diverse defensive mechanisms. It introduces a PSRO-style equilibrium computation framework for strategic agent interactions, and a novel zero-day exploit modeling approach. Its realism and analytic power are demonstrated via deployment against the Volt Typhoon APT scenario.

	  Paper: [(2025) CyGym: A Simulation-Based Game-Theoretic Analysis Framework for Cybersecurity](https://arxiv.org/pdf/2506.21688)

Cyborg++

     ![](imgs/cyborgplusplus.png) 
  

  

    [CybORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents](https://github.com/alan-turing-institute/CybORG_plus_plus)

CybORG++ is an advanced toolkit for reinforcement learning research focused on network defence. Building on the CAGE 2 CybORG environment, it introduces key improvements, including enhanced debugging capabilities, refined agent implementation support, and a streamlined environment that enables faster training and easier customization. Along with addressing several software bugs from its predecessor, CybORG++ introduces MiniCAGE, a lightweight version of CAGE 2.

	  Paper: [(2024) CybORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents](https://arxiv.org/pdf/2410.16324v1)

Cybershield

     ![](imgs/cybershield.png) 
  

  

    [CYBERSHIELD: A Competitive Simulation Environment for Training AI in Cybersecurity](https://ieeexplore.ieee.org/document/10710208)

CyberShield encompasses a comprehensive environment with multiple computers, each hosting various services with unique vulnerabilities. Within this environment, two opposing agents, defender and attacker, participate in a strategic battle, each equipped with distinct actions aimed at outsmarting the other. CyberShield is optimized for competitive multi-agent training using RL algorithms.

	  Paper: [(2024) CYBERSHIELD: A Competitive Simulation Environment for Training AI in Cybersecurity](https://ieeexplore.ieee.org/document/10710208)

Cyberwheel

     ![](imgs/cyberwheel.png) 
  

  

    [Cyberwheel: A Reinforcement Learning Simulation Environment](https://github.com/ORNL/cyberwheel)

Cyberwheel is a Reinforcement Learning (RL) simulation environment built for training and evaluating autonomous cyber defense models on simulated networks. It was built with modularity in mind, to allow users to build on top of it to fit their needs, supporting various robust configuration files to build networks, services, host types, defensive agents, and more. Cyberwheel is being developed by Oak Ridge National Lab (ORNL).

	  Paper: [(2024) Towards a High Fidelity Training Environment for Autonomous Cyber Defense Agents](https://doi.org/10.1145/3675741.3675752)

Pentesting Training Framework for Reinforcement Learning Agents (PenGym)

     ![](imgs/pengym.png) 
  

  

    [PenGym: Pentesting Training Framework for Reinforcement Learning Agents](https://github.com/cyb3rlab/PenGym)

PenGym is a framework for creating and managing realistic environments used for the training of Reinforcement Learning (RL) agents for penetration testing purposes. PenGym uses the same API with the Gymnasium fork of the OpenAI Gym library, thus making it possible to employ PenGym with all the RL agents that follow those specifications. PenGym is being developed by Japan Advanced Institute of Science and Technology (JAIST) in collaboration with KDDI Research, Inc.

	  Paper: [(2024) PenGym: Pentesting Training Framework for Reinforcement Learning Agents](https://www.jaist.ac.jp/~razvan/publications/pengym_framework_rl_agents.pdf)  

  
  
  Paper: [(2025) PenGym: Realistic training environment for reinforcement learning pentesting agents](https://www.sciencedirect.com/science/article/pii/S0167404824004450)  

  
  
  Thesis: [(2024) Realistic Pentesting Training Framework for Reinforcement Learning Agents](https://dspace.jaist.ac.jp/dspace/bitstream/10119/19357/5/paper.pdf)

The ARCD Primary-level AI Training Environment (PrimAITE)

     ![](imgs/primaite.png) 
  

  

    [The ARCD Primary-level AI Training Environment (PrimAITE)](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE)
    
      
      The ARCD Primary-level AI Training Environment (PrimAITE) provides an effective simulation capability for the purposes of training and evaluating AI in a cyber-defensive role.

AutoPentest-DRL

     ![](https://raw.githubusercontent.com/crond-jaist/AutoPentest-DRL/master/Figures/framework_overview.png) 
  

  

    [AutoPentest-DRL: Automated Penetration Testing Using Deep Reinforcement Learning](https://github.com/crond-jaist/AutoPentest-DRL)
    
      
        AutoPentest-DRL is an automated penetration testing framework based on Deep Reinforcement Learning (DRL) techniques. AutoPentest-DRL can determine the most appropriate attack path for a given logical network, and can also be used to execute a penetration testing attack on a real network via tools such as Nmap and Metasploit. This framework is intended for educational purposes, so that users can study the penetration testing attack mechanisms. AutoPentest-DRL is being developed by the Cyber Range Organization and Design ([CROND](https://www.jaist.ac.jp/misc/crond/index-en.html)) NEC-endowed chair at the Japan Advanced Institute of Science and Technology ([JAIST](https://www.jaist.ac.jp/english/)) in Ishikawa,Japan.

NASimEmu

     ![](imgs/nasimemu.svg) 
  

  

    [NASimEmu](https://github.com/jaromiru/NASimEmu)
    
      
        NASimEmu is a framework for training deep RL agents in offensive penetration-testing scenarios. It includes both a simulator and an emulator so that a simulation-trained agent can be seamlessly deployed in emulation. Additionally, it includes a random generator that can create scenario instances varying in network configuration and size while fixing certain features, such as exploits and privilege escalations. Furthermore, agents can be trained and tested in multiple scenarios simultaneously.  


        Paper: [(2023) NASimEmu: Network Attack Simulator & Emulator for Training Agents Generalizing to Novel Scenarios](https://arxiv.org/abs/2305.17246)  

        Framework: [NASimEmu](https://github.com/jaromiru/NASimEmu)  

        Implemented agents: [NASimEmu-agents](https://github.com/jaromiru/NASimEmu-agents)

gym-idsgame

     ![](gifs/gym_idsgame.gif) 
  

  

    [gym-idsgame](https://github.com/Limmen/gym-idsgame)
    
      
        An Abstract Cyber Security Simulation and Markov Game for OpenAI Gym.
        Paper: [(2020) Finding Effective Security Strategies through Reinforcement Learning and Self-Play](https://arxiv.org/abs/2009.08120)

CyberBattleSim (Microsoft)

     ![](imgs/cyberbattlesim_env.png) 
  

  

    [CyberBattleSim](https://github.com/microsoft/CyberBattleSim)
    
      
        CyberBattleSim is an experimentation research platform to investigate the interaction of automated agents operating in a simulated abstract enterprise network environment. The simulation provides a high-level abstraction of computer networks and cyber security concepts. Its Python-based Open AI Gym interface allows for the training of automated agents using reinforcement learning algorithms.
        Blogpost: [(2021) Gamifying machine learning for stronger security and AI models](https://www.microsoft.com/security/blog/2021/04/08/gamifying-machine-learning-for-stronger-security-and-ai-models/)

gym-malware

     ![](imgs/malware_env.png) 
  

  

    [gym-malware](https://github.com/endgameinc/gym-malware)
    
      
        Malware Env for OpenAI Gym
        Paper: [(2018) Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning](https://arxiv.org/pdf/1801.08917.pdf)

malware-rl

     ![](imgs/malware_env.png) 
  

  

    [malware-rl](https://github.com/bfilar/malware_rl)
    
      
        Extended and Updated `gym_malware` which supports recent LIEF versionS and an enhanced collection of models (EMBER, MalConv and SOREL-20M)
        Paper: [(2018) Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning](https://arxiv.org/pdf/1801.08917.pdf)

gym-flipit

     ![](imgs/flipit_env.png) 
  

  

    [gym-flipit](https://github.com/lisaoakley/gym-flipit)
    
      
        Gym environment for FLIPIT: The Game of "Stealthy Takeover" invented by Marten van Dijk, Ari Juels, Alina Oprea, and Ronald L. Rivest.
        Paper: [(2019) QFlip: An Adaptive Reinforcement Learning Strategy for the FlipIt Security Game](https://arxiv.org/abs/1906.11938)

gym-threat-defense

     ![](imgs/threat_defense_env.png) 
  

  

    [gym-threat-defense](https://github.com/hampusramstrom/gym-threat-defense)
    
      
        Gym environment for the environment described in the paper: [(2019) Optimal Defense Policies for Partially Observable Spreading Processes on Bayesian Attack Graphs](https://dl.acm.org/doi/10.1145/2808475.2808482)

gym-nasim

     ![](imgs/nasim_env.png) 
  

  

    [gym-nasim](https://github.com/Jjschwartz/NetworkAttackSimulator)
    
      
        Thesis: [(2018) Autonomous Penetration Testing using Reinforcement Learning](https://arxiv.org/pdf/1905.05965.pdf)

gym-optimal-intrusion-response

     ![](imgs/intrusion_response_env.png) 
  

  

    [gym-optimal-intrusion-response](https://github.com/Limmen/gym-optimal-intrusion-response)
    
      
      An OpenAI Gym interface to a MDP/Markov Game model for optimal intrusion response of a realistic infrastructure simulated using system traces.
      Paper: [(2021) Learning Intrusion Prevention Policies through Optimal Stopping](https://arxiv.org/pdf/2106.07160.pdf)

sql_env

     ![](imgs/sql_env.png) 
  

  

    [sql_env](https://github.com/manuel-delverme/sql_env)
    
      
      Paper: [(2021) SQL Injections and Reinforcement Learning: An Empirical Evaluation of the Role of Action Structure](https://link.springer.com/chapter/10.1007/978-3-030-91625-1_6)

cage-challenge

     ![](imgs/cage_env.png) 
  





  

    [cage-challenge-1](https://github.com/cage-challenge/cage-challenge-1)
    
      
       The first Cyber Autonomos Gym for Experimentation (CAGE) challenge environment released at the 1st International Workshop on Adaptive Cyber Defense held as part of the 2021 International Joint Conference on Artificial Intelligence (IJCAI).
      
    
  







    [cage-challenge-2](https://github.com/cage-challenge/cage-challenge-2)
    
      
       The second Cyber Autonomous Gym for Experimentation (CAGE) challenge environment announced at the AAAI-22 Workshop on Artificial Intelligence for Cyber Security Workshop (AICS).
	   Paper: [(2023) On Autonomous Agents in a Cyber Defence Environment](https://arxiv.org/pdf/2309.07388)
      
    
  







    [cage-challenge-3](https://github.com/cage-challenge/cage-challenge-3)
    
      
       The third Cyber Autonomous Gym for Experimentation (CAGE) challenge environment.
      
    
  







    [cage-challenge-4](https://github.com/cage-challenge/cage-challenge-4)
    
      
       The fourth Cyber Autonomous Gym for Experimentation (CAGE) challenge environment.

ATMoS

     ![](imgs/atmos.png) 
  

  

    [ATMoS](https://github.com/ATMoS-Waterloo/ATMoS)
    
      
      Paper: [(2020) ATMoS: Autonomous Threat Mitigation in SDN using Reinforcement Learning](https://ieeexplore.ieee.org/document/9110426)

MAB-Malware

     ![](imgs/mab_malware.png) 
  

  

    [MAB-malware](https://github.com/weisong-ucr/MAB-malware)
    
      
      Paper: [(2022) MAB-Malware: A Reinforcement Learning Framework for Attacking Static Malware Classifiers](https://arxiv.org/pdf/2003.03100.pdf)

ASAP

     ![](imgs/asap.png) 
  

  

    [Autonomous Security Analysis and Penetration Testing framework (ASAP)](https://github.com/ankur8931/asap)
    
      
      Paper: [(2020) Autonomous Security Analysis and Penetration Testing](https://ieeexplore.ieee.org/document/9394285)

Yawning Titan

     ![](imgs/yawning_titan_env.png) 
  

  

    [Yawning Titan](https://github.com/dstl/YAWNING-TITAN)
    
      
        Yawning Titan is an abstract, highly flexible, cyber security simulator that is capable of simulating a range of cyber security scenarios.

Paper: (2022) Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulation

Cyborg

     ![](imgs/cyborg.png) 
  

  

    [Cyborg](https://github.com/cage-challenge/CybORG)
    
      
        Cyborg is a gym for autonomous cyberg operations research that is driven by the need to efficiently support reinforcement learning to train adversarial decision-making models through simulation and emulation. This is a variation of the environments used by cage-challenge above.

Paper: (2021) CybORG: A Gym for the Development of Autonomous Cyber Agents

FARLAND

     ![](imgs/farland.png) 
  

  

    FARLAND (github repository missing)
    
      
        FARLAND is a framework for advanced Reinforcement Learning for autonomous network defense, that uniquely enables the design of network environments to gradually increase the complexity of models, providing a path for autonomous agents to increase their performance from apprentice to superhuman level, in the task of reconfiguring networks to mitigate cyberattacks.

Paper: (2021) Network Environment Design for Autonomous Cyberdefense

SecureAI

     ![](imgs/secureai.png) 
  

  

    [SecureAI](https://github.com/ansi-code/secureai-java)
    
      
        SecureAI: Deep Reinforcement Learning for Self-Protection in Non-Stationary Cloud Architectures
		Paper: [(2021) An Intrusion Response Approach for Elastic Applications Based on Reinforcement Learning](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9659882)

CYST

     ![](imgs/cyst.png) 
  

  

    [CYST](https://muni.cz/go/cyst-user)
    
      
        CYST is a multi-agent discrete-event simulation framework tailored for cybersecurity domain. Its goal is to enable high-throughput and realistic simulation of cybersecurity interactions in arbitrary infrastructures.   


		Paper: [(2020) Session-level Adversary Intent-Driven Cyberattack Simulator](https://ieeexplore.ieee.org/abstract/document/9213690)
  Code: [HERE](https://gitlab.ics.muni.cz/cyst-public/cyst-core/)

CLAP

     ![](imgs/clap.jpg) 
  

  

    [CLAP: Curiosity-Driven Reinforcment Learning Automatic Penetration Testing Agent](https://github.com/yyzpiero/RL4RedTeam)
    
      
        CLAP is a reinforcement learning PPO agent performs Penetration Testing in simulated computer network environment (we use Network Attack Simulator (NASim)). The agent is trained to scan for vulnerabilities in the network and exploit them to gain access to various network resources.    


		Paper: [(2022) Behaviour-Diverse Automatic Penetration Testing: A Curiosity-Driven Multi-Objective Deep Reinforcement Learning Approach](https://arxiv.org/abs/2202.10630)
  Code: [HERE](https://github.com/yyzpiero/RL4RedTeam)

CyGIL

     ![](imgs/cygil.png) 
  

  

    [CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems](https://arxiv.org/abs/2109.03331)
    
      
        CyGIL is an experimental testbed of an emulated RL training environment for network cyber operations. CyGIL uses a stateless environment architecture and incorporates the MITRE ATT&CK framework to establish a high fidelity training environment, while presenting a sufficiently abstracted interface to enable RL training. Its comprehensive action space and flexible game design allow the agent training to focus on particular advanced persistent threat (APT) profiles, and to incorporate a broad range of potential threats and vulnerabilities. By striking a balance between fidelity and simplicity, it aims to leverage state of the art RL algorithms for application to real-world cyber defence.   


		Paper: [(2021) CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems](https://arxiv.org/abs/2109.03331)

BRAWL

     ![](imgs/brawl.png) 
  

  

    [BRAWL](https://github.com/mitre/brawl-public-game-001)
    
      
        BRAWL seeks to create a compromise by creating a system to automatically create an enterprise network inside a cloud environment. OpenStack is the only currently supported environment, but it is being designed in such a way as to easily support other cloud environments in the future.

DETERLAB

     ![](imgs/deter.jpeg) 
  

  

    [DeterLab: Cyber-Defense Technology Experimental Research Laboratory](https://ieeexplore.ieee.org/document/5655108)
    
      
        Since 2004, the DETER Cybersecurity Testbed Project has worked to create the necessary infrastructure - facilities, tools, and processes-to provide a national resource for experimentation in cyber security. The next generation of DETER envisions several conceptual advances in testbed design and experimental research methodology, targeting improved experimental validity, enhanced usability, and increased size, complexity, and diversity of experiments.   


		Paper: [(2010) The DETER project: Advancing the science of cyber security experimentation and test](https://ieeexplore.ieee.org/document/5655108)

EmuLab

     ![]('imgs/emulab.png) 
  

  

    [EmuLab: Large-scale Virtualization in the Emulab Network Testbed]('https://www.usenix.org/legacy/event/usenix08/tech/full_papers/hibler/hibler.pdf)

The Emulab software is the management system for a network-rich PC cluster that provides a space- and timeshared public facility for studying networked and distributed systems.

		Paper: [(2008) EmuLab: Large-scale Virtualization in the Emulab Network Testbed](https://www.usenix.org/legacy/event/usenix08/tech/full_papers/hibler/hibler.pdf)

Mininet

     ![](imgs/emulab.svg) 
  

  

    [Mininet creates a realistic virtual network, running real kernel, switch and application code, on a single machine (VM, cloud or native), in seconds, with a single command.](https://ieeexplore.ieee.org/document/7311238)
    
      
		Paper: [(2015) Emulation of Software Defined Networks Using Mininet in Different Simulation Environments](https://ieeexplore.ieee.org/document/7311238)

Vine

     ![](imgs/vine.png) 
  

  

    [VINE: A Cyber Emulation Environment for MTD Experimentation](https://dl.acm.org/doi/10.1145/2808475.2808486)
    
      
		Paper: [(2015) VINE: A Cyber Emulation Environment for MTD Experimentation](https://dl.acm.org/doi/10.1145/2808475.2808486)

CRATE

     ![](imgs/crate.png) 
  

  

    [CRATE Exercise Control – A cyber defense exercise management and support

tool](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9229649&casa_token=b3Vk13dXj_sAAAAA:zrB-U_pA50aq7IMzcdy6qy9YNjFsuccAtpujDmDvJnNq9iCc7aqQNoeKsjC_AddSIgYt-MUkk3A6)

		Paper: [(2020) CRATE Exercise Control – A cyber defense exercise management and support](https://dl.acm.org/doi/10.1145/2808475.2808486)

GALAXY

     ![](imgs/galaxy.png) 
  

  

    [Galaxy: A Network Emulation Framework for Cybersecurity

tool](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9229649&casa_token=b3Vk13dXj_sAAAAA:zrB-U_pA50aq7IMzcdy6qy9YNjFsuccAtpujDmDvJnNq9iCc7aqQNoeKsjC_AddSIgYt-MUkk3A6)

		Paper: [(2018) Galaxy: A Network Emulation Framework for Cybersecurity](https://www.usenix.org/system/files/conference/cset18/cset18-paper-schoonover.pdf)

↑ Papers

Surveys

Demonstration papers

Position papers

Regular Papers

PhD Theses

Master Theses

Bachelor Theses

Posters

↑ Books

↑ Blogposts

↑ Talks

排名相邻
同类项目