LLM-Agent-Survey

A Reading List for LLM-Agents (Last Major Updated: 14 Mar 2025)

Xinzhe Li

[![Paper](https://img.shields.io/badge/CoLing-2025-b31b1b)](https://arxiv.org/abs/2406.05804)
[![Github](https://img.shields.io/github/last-commit/xinzhel/llm-agent-survey?color=blue)](https://github.com/xinzhel/llm-agent-survey)
[![License](https://img.shields.io/github/license/xinzhel/llm-agent-survey?color=green)](https://github.com/xinzhel/llm-agent-survey/blob/main/LICENSE)

This Repository vs. Others

Our Github Repository follows the selection criteria below:

Allowing Coherent Understanding: They can be systematically categoried into the unified framework in my survey, according to the use of LLM-Profiled Roles (LMPRs).
- A general survey (Accepted at CoLing 2025): A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning
High Quality: Papers are published on ICML, ICLR, NeurIPS, *ACL (including EMNLP), and COLING. Or unpublished papers contain useful analysis and insightful novelty
- Unpublished papers are marked with 💡 and will be updated upon publication.
- Paper Reviews: The paper links to OpenReview (if available) are alwasy given. I often learn much more from and resonate with many reviews about the papers and evaluate some rejected papers with the reviews. (That's why I always like NeurIPS/ICLR papers).
Exhasutive Review on Search Workflows
- A corresponding survey on TMLR 2025: A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworks

From Apr 2025, I will not actively update this repo since my recent research focuses on LLM Inference via Search or LIS. But I am sure that you can follow some actively updated repos below for the latest papers. Here is some reason for my transfer: 1) the LIS algorithms/workflows contain less heuristic, delibrate design, and human-aligned slow thinking while generally applicable. 2) with the limited energy as an individual researcher on inference-time workflows/agents, I do not have enough energy to discover the details of newly published work in this general field. Putting all the energy on LIS can help me maintain the quality of my work delivered to the community. Hopefully I can find a professional team later to work more ambitiously.

Other Github Repositories summarize related papers with less constrained selection criteria:

Other Github Repositories summarize related papers focusing on specific perspectives:

LLM-Search
nuster1128/LLM_Agent_Memory_Survey : Focus on memory
teacherpeterpan/self-correction-llm-papers : Focus on feedback learning (Self Correction)
git-disl/awesome-LLM-game-agent-papers : Focus on gaming applications

🎁 Surveys
🚀 Tool Use
🧠 Planning
🔄 Feedback Learning
🧩 Composition
- Planning + Feedback Learning
- Planning + Tool Use
🌍 World Modeling
📊 Benchmarks
📝 Citation

:gift: Surveys

A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning, CoLing 2025 [paper]
A Survey on Large Language Model based Autonomous Agents, Frontiers of Computer Science 2024 [paper] | [code]
Augmented Language Models: a Survey, TMLR [paper]
Understanding the planning of LLM agents: A survey, arXiv [paper] 💡
The Rise and Potential of Large Language Model Based Agents: A Survey, arxiv [paper] 💡
A Survey on the Memory Mechanism of Large Language Model based Agents, arxiv [paper] 💡

:rocket: Tool Use

ReAct: Synergizing Reasoning and Acting in Language Models, ICLR 2023 [paper]
Toolformer: Language Models Can Teach Themselves to Use Tools, NeurIPS 2023 [paper]
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face, NeurIPS 2023 [paper]
API-Bank: A Benchmark for Tool-Augmented LLMs, EMNLP 2023 [paper]
ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings, NeurIPS 2023 [paper]
MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Prompting, ACL 2023 [paper]
ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models, EMNLP 2023 [paper]
ART: Automatic multi-step reasoning and tool-use for large language models, arXiv.2303.09014 [paper] 💡
TALM: Tool Augmented Language Models, arXiv.2205.12255 [paper] 💡
On the Tool Manipulation Capability of Open-source Large Language Models, arXiv.2305.16504 [paper] 💡
Large Language Models as Tool Makers, arXiv.2305.17126 [paper] 💡
GEAR: Augmenting Language Models with Generalizable and Efficient Tool Resolution, arXiv.2307.08775 [paper] 💡
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs, arXiv.2307.16789 [paper] 💡
Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models, arXiv.2308.00675 [paper] 💡
MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback, arXiv.2309.10691 [paper] 💡
Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning, arXiv.2309.10814 [paper] 💡

:brain: Planning

Base Workflows

On the Planning Abilities of Large Language Models -- A Critical Investigation, NeurIPS 2023 [paper]

Search Workflows

Details in the page (on the way to be publised).

Alphazero-like Tree-Search can guide large language model decoding and training, ICML 2024 [paper]
- Search Algorithm: MCTS
Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models, ICML 2024 [paper]
- Search Algorithm: MCTS
When is Tree Search Useful for {LLM} Planning? It Depends on the Discriminator, ACL 2024 [paper]
- Search Algorithm: MCTS
Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation, ACL findings 2024 [paper]
- Search Algorithm: MCTS
Tree-of-Traversals: A Zero-Shot Reasoning Algorithm for Augmenting Black-box Language Models with Knowledge Graphs, ACL 2024 [paper]
- Search Algorithm: BFS/DFS
LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path Planning, EMNLP findings 2024 [paper] | [code]
- Search Algorithm: A*
LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models, COLM2024 [paper] | [code]
Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models, arXiv.2310.04406 [paper] 💡
Large Language Model Guided Tree-of-Thought, arXiv.2305.08291 [paper]💡
Tree Search for Language Model Agents, Under Review [paper]💡
- Search Algorithm: Best-First Search
Q*: Improving multi-step reasoning for llms with deliberative planning, Under Review [paper]💡
- Search Algorithm: A*
Planning with Large Language Models for Code Generation, ICLR 2023 [paper]
- Search Algorithm: MCTS
Tree of Thoughts: Deliberate Problem Solving with Large Language Models, NeurIPS 2023 [paper]
- Search Algorithm: BFS/DFS
LLM-MCTS:Large Language Models as Commonsense Knowledge for Large-Scale Task Planning, NeurIPS 2023 [paper] | [code]
- Search Algorithm: MCTS
Self-Evaluation Guided Beam Search for Reasoning, NeurIPS 2023 [paper]
- Search Algorithm: BFS/DFS
PathFinder: Guided Search over Multi-Step Reasoning Paths, NeurIPS 2023 R0-FoMo [paper]
- Search Algorithm: Beam Search
Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts, EMNLP 2023 [paper]
RAP: Reasoning with Language Model is Planning with World Model, EMNLP 2023 [paper]
- Search Algorithm: MCTS
Prompt-Based Monte-Carlo Tree Search for Goal-oriented Dialogue Policy Planning, EMNLP 2023 [paper]
- Search Algorithm: MCTS
Monte Carlo Thought Search: Large Language Model Querying for Complex Scientific Reasoning in Catalyst Design, EMNLP findings 2023 [paper]
- Search Algorithm: MCTS
Agent q: Advanced reasoning and learning for autonomous ai agents, arXiv.2309.10814 [paper] 💡
- Search Algorithm: MCTS

Decomposition

HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face, NeurIPS 2023 [paper] | [code]
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models, NeurIPS 2023 [paper]

PDDL+Local Search

Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning, NeurIPS 2023 [paper] | [code]
On the Planning Abilities of Large Language Models - A Critical Investigation, NeurIPS 2023 [paper] | [code]
PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change, NeurIPS 2023 [paper] | [code]

Others

LLM+P: Empowering Large Language Models with Optimal Planning Proficiency, arXiv.2304.11477 [paper]💡

:arrows_counterclockwise: Feedback Learning

Reflexion: Language Agents with Verbal Reinforcement Learning, NeurIPS 2023 [paper]
Self-Refine: Iterative Refinement with Self-Feedback, NeurIPS 2023 [paper]
SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning, ICLR 2024 [paper] | [code]
Learning From Correctness Without Prompting Makes LLM Efficient Reasoner, COLM2024 [paper]
Learning From Mistakes Makes LLM Better Reasoner, arXiv [paper] | [code]💡
LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback ACL 2024 [paper]

:jigsaw: Composition

Planning + Feedback Learning

AdaPlanner: Adaptive Planning from Feedback with Language Models, NeurIPS 2023 [paper]
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing, ICLR 2024 [paper]
ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning, arXiv.2308.13724 [paper] 💡

Planning + Tool Use

ToolChain: Efficient Action Space Navigation in Large Language Models with A* Search, ICLR 2024 [paper]
TPTU: Task Planning and Tool Usage of Large Language Model-based AI Agents, FMDM @ NeurIPS 2023 [paper]
TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems, LLMAgents @ ICLR 2024 [paper]

:world_map: World Modeling

Can Language Models Serve as Text-Based World Simulators?, ACL 2024 [paper] | [code]
Making Large Language Models into World Models with Precondition and Effect Knowledge, arXiv [paper] 💡
Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning, NeurIPS 2023 [paper] | [code]
ByteSized32: A Corpus and Challenge Task for Generating Task-Specific World Models Expressed as Text Games, EMNLP 2023 [paper] | [code]

:bar_chart: Benchmarks

Tool-Use Benchmarks

MetaTool Benchmark: Deciding Whether to Use Tools and Which to Use, arXiv.2310.03128 [paper] 💡
TaskBench: Benchmarking Large Language Models for Task Automation, arXiv.2311.18760 [paper] 💡

Planning Benchmarks

Large Language Models Still Can't Plan (A Benchmark for LLMs on Planning and Reasoning about Change), NeurIPS 2023 [paper]

:star: Star History

:memo: Citation

If you find our work helpful, you can cite this paper as:

@inproceedings{li-2025-review,
    title = "A Review of Prominent Paradigms for {LLM}-Based Agents: Tool Use, Planning (Including {RAG}), and Feedback Learning",
    author = "Li, Xinzhe",
    editor = "Rambow, Owen  and
      Wanner, Leo  and
      Apidianaki, Marianna  and
      Al-Khalifa, Hend  and
      Eugenio, Barbara Di  and
      Schockaert, Steven",
    booktitle = "Proceedings of the 31st International Conference on Computational Linguistics",
    month = jan,
    year = "2025",
    address = "Abu Dhabi, UAE",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.coling-main.652/",
    pages = "9760--9779",
}

@article{li2025a,
title={A Survey on {LLM} Test-Time Compute via Search: Tasks, {LLM} Profiling, Search Algorithms, and Relevant Frameworks},
author={Xinzhe Li},
journal={Transactions on Machine Learning Research},
issn={2835-8856},
year={2025},
url={https://openreview.net/forum?id=x9VQFjtOPS},
note={}
}