Verl-Tool
VerlTool: A unified and easy-to-extend tool-agent training framework based on verl.
| Paper | Quick Start | Training Recipes | DeepWiki | WeChat Group | Discord |
News
- [2026/06/01] 🏆 Our paper has been accepted by TMLR 2026!
- [2026/05/01] 🏆 Our paper received the Best Paper Award at ICLR 2026 SPOT!
- [2025/11/10] VerlTool has re-organized its codebase to improve modularity and maintainability, supporting to the latest verl (
0.6.0) and vllm (0.11.0) versions. Please refer to the verl-tool v0.6.0.dev Upgrade Notes for more details. - [2025/09/02] VerlTool's tech report is out! See on Hugging Face Daily Paper!
- [2025/06/30] We reproduce Search-R1 with even higher performance on the same benchmarks! See PR and training README for more details.
- [2025/06/28] We support NL2SQL tool RL training. See NL2SQL README for more details.
- [2025/06/26] We support DAPO recipe training. See DAPO.md for more details.
- [2025/06/18] VerlTool now officially supports Trajectory-Level asynchronous, speeding up the rollout generation with tool calling by at least 2x! see asyncRL.md for more details.
- [2025/06/16] We have updated the verl submodule to the latest version (06/16) and modified some code to adapt to the new version.
- [2025/06/13] We integrated DeepWiki for Verl-Tool. Feel free to browse the AI-generated docs and chat with Verl-tool codes.
- [2025/06/06] We have updated a detailed design overview in the README, including how to add new tools, how to use the tool server, and how to train your own models with verl-tool.
- [2025/05/31] We released the Verl-tool training/evaluation code with ToRL training as an initial example (see X post). We are working on the paper and will release it very soon.
Features
- 🔧 Complete decoupling of actor rollout and environment interaction - We use verl as a submodule to benefit from ongoing verl repository updates. All tool calling is integrated via a unified API, allowing you to easily add new tools by simply adding a Python file and testing independently.
- 🌍 Tool-as-environment paradigm - Each tool interaction can modify the environment state. We store and reload environment states for each trajectory.
- ⚡ Native RL framework for tool-calling agents - verl-tool natively supports multi-turn interactive loops between agents and their tool environments.
- 📊 User-friendly evaluation suite - Launch your trained model with OpenAI API alongside the tool server. Simply send questions and get final outputs with all interactions handled internally. See benchmarks.
📚 Contents Link
- 📖 Installation Guide
- ⚡ Synchronous Rollout Design
- 🔄 Asynchronous Rollout Design
- 🛠️ Tool Server Design
- 🎯 Training Guide
- 📊 Evaluation Guide
- 🔧 Update Verl Submodule Version
- 📈 Existing Training Results
- 🤝 Contributing Guide
Core Contributors
[
Dongfu Jiang](https://github.com/jdf-prog)
[
Zhuofeng Li](https://github.com/Zhuofeng-Li)
[
Yi Lu](https://github.com/EigenTom)
[
Zhiheng Lvu](https://github.com/cogito233)
[
Ping Nie](https://github.com/erenup)
Advisors
[
Wenhu Chen](https://github.com/wenhuchen)
[
Tianyu Pang](https://github.com/P2333)
[
Chao Du](https://github.com/duchao0726)
Acknowledgements
We thank the following open-source projects for making verl-tool possible:
- VLLM and SGLang for their fast LLM inference support!
- verl for the excellent RL framework design.
- SearchR1, RAGEN, and ToRL for their early-stage exploration of tool-agent RL training.
We thank Netmind.AI, SeaAI Lab, and Map for GPU support!
Community Projects Inspired by Verl-Tool
- AgentFlow: In-the-Flow Agentic System Optimization
Star History
Badge
Citation
@article{jiang2025verltool,
title={VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use},
author={Jiang, Dongfu and Lu, Yi and Li, Zhuofeng and Lyu, Zhiheng and Nie, Ping and Wang, Haozhe and Su, Alex and Chen, Hui and Zou, Kai and Du, Chao and others},
journal={arXiv preprint arXiv:2509.01055},
year={2025}
}