AI Rank
中文
← Open Source
reasoning-survey

Awesome-Reasoning-Foundation-Models

✨✨Latest Papers and Benchmarks in Reasoning with Foundation Models.

ListsPaper collections
Open on GitHub
Momentum
+0stars in 24 hours0.0%
660
Stars
64
Forks
+0
This week
13
Contributors
Created 2023-11-21 · Updated 2026-10-04 · #15359 today
Top developers
chuanyang-Zhengjiaqixuacreasoning-survey
README

Awesome-Reasoning-Foundation-Models

survey.pdf | A curated list of awesome large AI models, or foundation models, for reasoning.

We organize the current foundation models into three categories: language foundation models, vision foundation models, and multimodal foundation models. Further, we elaborate the foundation models in reasoning tasks, including commonsense, mathematical, logical, causal, visual, audio, multimodal, agent reasoning, etc. Reasoning techniques, including pre-training, fine-tuning, alignment training, mixture of experts, in-context learning, and autonomous agent, are also summarized.

We welcome contributions to this repository to add more resources. Please submit a pull request if you want to contribute! See CONTRIBUTING.

Table of Contents

table of contents

0 Survey

This repository is primarily based on the following paper:

A Survey of Reasoning with Foundation Models: Concepts, Methodologies, and Outlook

[Paper][ArXiv]

Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, Yue Wu, Wenhai Wang, Junsong Chen, Zhangyue Yin, Xiaozhe Ren, Jie Fu, Junxian He, Wu Yuan, Qi Liu, Xihui Liu, Yu Li, Hao Dong, Yu Cheng, Ming Zhang, Pheng Ann Heng, Jifeng Dai, Ping Luo, Jingdong Wang, Ji-Rong Wen, Xipeng Qiu, Yike Guo, Hui Xiong, Qun Liu, and Zhenguo Li

If you find this repository helpful, please consider citing:

@article{sun2025survey,
  author = {Sun, Jiankai and Zheng, Chuanyang and Xie, Enze and Liu, Zhengying and Chu, Ruihang and Qiu, Jianing and Xu, Jiaqi and Ding, Mingyu and Li, Hongyang and Geng, Mengzhe and Wu, Yue and Wang, Wenhai and Chen, Junsong and Yin, Zhangyue and Ren, Xiaozhe and Fu, Jie and He, Junxian and Wu, Yuan and Liu, Qi and Liu, Xihui and Li, Yu and Dong, Hao and Cheng, Yu and Zhang, Ming and Heng, Pheng Ann and Dai, Jifeng and Luo, Ping and Wang, Jingdong and Wen, Ji-Rong and Qiu, Xipeng and Guo, Yike and Xiong, Hui and Liu, Qun and Li, Zhenguo},
  title = {A Survey of Reasoning with Foundation Models: Concepts, Methodologies, and Outlook},
  year = {2025},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  issn = {0360-0300},
  url = {https://doi.org/10.1145/3729218},
  doi = {10.1145/3729218},
  abstract = {Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation. It serves as a fundamental methodology in the field of Artificial General Intelligence (AGI). With the ongoing development of foundation models, there is a growing interest in exploring their abilities in reasoning tasks. In this paper, we introduce seminal foundation models proposed or adaptable for reasoning, highlighting the latest advancements in various reasoning tasks, methods, and benchmarks. We then delve into the potential future directions behind the emergence of reasoning abilities within foundation models. We also discuss the relevance of multimodal learning, autonomous agents, and super alignment in the context of reasoning. By discussing these future research directions, we hope to inspire researchers in their exploration of this field, stimulate further advancements in reasoning with foundation models, e.g. Large Language Models (LLMs), and contribute to the development of AGI.},
  journal = {ACM Comput. Surv.},
  month = apr,
  keywords = {Reasoning, Foundation Models, Multimodal, AI Agent, Artificial General Intelligence, LLM}
}

1 Relevant Surveys and Links

relevant surveys

(Back-to-Top)

  • Combating Misinformation in the Age of LLMs: Opportunities and Challenges - [arXiv] [Link]

  • The Rise and Potential of Large Language Model Based Agents: A Survey - [arXiv] [Link]

  • Multimodal Foundation Models: From Specialists to General-Purpose Assistants - [arXiv] [Tutorial]

  • A Survey on Multimodal Large Language Models - [arXiv] [Link]

  • Interactive Natural Language Processing - [arXiv] [Link]

  • A Survey of Large Language Models - [arXiv] [Link]

  • Self-Supervised Multimodal Learning: A Survey - [arXiv] [Link]

  • Large AI Models in Health Informatics: Applications, Challenges, and the Future - [arXiv] [Paper] [Link]

  • Towards Reasoning in Large Language Models: A Survey - [arXiv] [Paper] [Link]

  • Reasoning with Language Model Prompting: A Survey - [arXiv] [Paper] [Link]

  • Awesome Multimodal Reasoning - [Link]

2 Foundation Models

foundation models

(Back-to-Top)

Table of Contents - 2

foundation models (table of contents)

(Back-to-Top)

2.1 Language Foundation Models

LFMs

Foundation Models (Back-to-Top)

2.2 Vision Foundation Models

VFMs

Foundation Models (Back-to-Top)

  • 2024/01 | Depth Anything | Yang et al.

Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data

[arXiv] [paper] [code] [project]

  • 2023/05 | SAA+ | Cao et al.

Segment Any Anomaly without Training via Hybrid Prompt Regularization

[arXiv] [paper] [code]

Segment Anything

[arXiv] [paper] [code] [blog]

Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

[arXiv] [paper] [code]

High-Resolution Image Synthesis with Latent Diffusion Models

[arXiv] [paper] [code] [stable diffusion ]

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

[arXiv] [paper] [code]

  • 2020/10 | ViT | Dosovitskiy et al., ICLR 2021

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

[arXiv] [paper] [Implementation]

2.3 Multimodal Foundation Models

MFMs

Foundation Models (Back-to-Top)

  • 2024/01 | LLaVA-1.6 | Liu et al.

LLaVA-1.6: Improved reasoning, OCR, and world knowledge

[code] [blog]

  • 2024/01 | MouSi | Fan et al.

MouSi: Poly-Visual-Expert Vision-Language Models

[arXiv] [paper] [code]

  • 2023/12 | InternVL | Chen et al.

InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks

[arXiv] [paper] [code]

Improved Baselines with Visual Instruction Tuning
[arXiv] [paper] [code] [project]

2.4 Reasoning Applications

reasoning applications

Foundation Models (Back-to-Top)

3 Reasoning Tasks

reasoning tasks

(Back-to-Top)

Table of Contents - 3

reasoning tasks (table of contents)

3.1 Commonsense Reasoning

commonsense reasoning

Reasoning Tasks (Back-to-Top)

3.1.1 Commonsense Question and Answering (QA)

3.1.2 Physical Commonsense Reasoning

3.1.3 Spatial Commonsense Reasoning

  • 2024/01 | SpatialVLM | Chen et al.

SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities

[arXiv] [paper] [project]

3.1.x Benchmarks, Datasets, and Metrics

3.2 Mathematical Reasoning

mathematical reasoning

Reasoning Tasks (Back-to-Top)

3.2.1 Arithmetic Reasoning

Mathematical Reasoning (Back-to-Top)

3.2.2 Geometry Reasoning

Mathematical Reasoning (Back-to-Top)

3.2.3 Theorem Proving

Mathematical Reasoning (Back-to-Top)

3.2.4 Scientific Reasoning

Mathematical Reasoning (Back-to-Top)

3.2.x Benchmarks, Datasets, and Metrics

Mathematical Reasoning (Back-to-Top)

  • 2024/01 | MathBench

MathBench: A Comprehensive Multi-Level Difficulty Mathematics Evaluation Dataset

[code]

3.3 Logical Reasoning

logical reasoning

Reasoning Tasks (Back-to-Top)

3.3.1 Propositional Logic

  • 2022/09 | Propositional Reasoning via Neural Transformer Language Models - [Paper]

3.3.2 Predicate Logic

3.3.x Benchmarks, Datasets, and Metrics

3.4 Causal Reasoning

causal reasoning

Reasoning Tasks (Back-to-Top)

3.4.1 Counterfactual Reasoning

3.4.x Benchmarks, Datasets, and Metrics

3.5 Visual Reasoning

visual reasoning

Reasoning Tasks (Back-to-Top)

3.5.1 3D Reasoning

3.5.x Benchmarks, Datasets, and Metrics

3.6 Audio Reasoning

audio reasoning

Reasoning Tasks (Back-to-Top)

3.6.1 Speech

3.6.x Benchmarks, Datasets, and Metrics

3.7 Multimodal Reasoning

multimodal reasoning

Reasoning Tasks (Back-to-Top)

3.7.1 Alignment

3.7.2 Generation

3.7.3 Multimodal Understanding

3.7.x Benchmarks, Datasets, and Metrics

3.8 Agent Reasoning

agent reasoning

Reasoning Tasks (Back-to-Top)

AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents

[arXiv] [paper] [project]

PaLM-E: An Embodied Multimodal Language Model

[arXiv] [paper] [project]

3.8.1 Introspective Reasoning

3.8.2 Extrospective Reasoning

3.8.3 Multi-agent Reasoning

3.8.4 Driving Reasoning

3.8.x Benchmarks, Datasets, and Metrics

3.9 Other Tasks and Applications

other tasks and applications

Reasoning Tasks (Back-to-Top)

3.9.1 Theory of Mind (ToM)

3.9.2 LLMs for Weather Prediction

  • 2022/09 | MetNet-2 | Deep learning for twelve hour precipitation forecasts - [Paper]

  • 2023/07 | Pangu-Weather | Accurate medium-range global weather forecasting with 3D neural networks - [Paper]

3.9.3 Abstract Reasoning

3.9.4 Defeasible Reasoning

3.9.5 Medical Reasoning

  • 2024/01 | CheXagent / CheXinstruct / CheXbench | Chen et al.

CheXagent: Towards a Foundation Model for Chest X-Ray Interpretation

[arXiv] [paper] [code] [project] [huggingface]

  • 2024/01 | EchoGPT | Chao et al.

EchoGPT: A Large Language Model for Echocardiography Report Summarization

[medRxiv] [paper]

  • 2023/10 | GPT4V-Medical-Report | Yan et al.

Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V

[arXiv] [paper] [code]

  • 2023/10 | VisionFM | Qiu et al.

VisionFM: a Multi-Modal Multi-Task Vision Foundation Model for Generalist Ophthalmic Artificial Intelligence

[arXiv] [paper]

  • 2023/09 | Yang et al.

The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)

[arXiv] [paper]

  • 2023/09 | RETFound | Zhou et al., Nature

A foundation model for generalizable disease detection from retinal images

[paper] [code]

  • 2023/08 | ELIXR | Xu et al.

ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders

[arXiv] [paper]

  • 2023/07 | Med-Flamingo | Moor et al.

Med-Flamingo: a Multimodal Medical Few-shot Learner

[arXiv] [paper] [code]

  • 2023/07 | Med-PaLM M | Tu et al.

Towards Generalist Biomedical AI

[arXiv] [paper] [code]

  • 2023/06 | Endo-FM | Wang et al., MICCAI 2023

Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train

[arXiv] [paper] [code]

  • 2023/06 | XrayGPT | Thawkar et al.

XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models

- [arXiv] [paper] [code]

  • 2023/06 | LLaVA-Med | Li et al., NeurIPS 2023

LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

[arXiv] [paper] [code]

  • 2023/05 | HuatuoGPT | Zhang et al., Findings of EMNLP 2023

HuatuoGPT, Towards Taming Language Model to Be a Doctor

[arXiv] [paper] [code]

  • 2023/05 | Med-PaLM 2 | Singhal et al.

Towards Expert-Level Medical Question Answering with Large Language Models

[arXiv] [paper]

  • 2022/12 | Med-PaLM / MultiMedQA / HealthSearchQA | Singhal et al., Nature

Large Language Models Encode Clinical Knowledge

[arXiv] [paper]

3.9.6 Bioinformatics Reasoning

3.9.7 Long-Chain Reasoning

4 Reasoning Techniques

reasoning techniques

(Back-to-Top)

Table of Contents - 4

reasoning techniques (table of contents)

4.1 Pre-Training

pre-training

Reasoning Techniques (Back-to-Top)

4.1.1 Data

a. Data - Text
b. Data - Image
c. Data - Multimodality

4.1.2 Network Architecture

a. Encoder-Decoder
b. Decoder-Only
c. CLIP Variants
d. Others

4.2 Fine-Tuning

fine-tuning

Reasoning Techniques (Back-to-Top)

4.2.1 Data

4.2.2 Parameter-Efficient Fine-tuning

a. Adapter Tuning
b. Low-Rank Adaptation
  • 2023/09 | LongLoRA / LongAlpaca-12k | Chen et al., ICLR 2024

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models

[arXiv] [paper] [code]

c. Prompt Tuning
d. Partial Parameter Tuning
e. Mixture-of-Modality Adaption

4.3 Alignment Training

alignment training

Reasoning Techniques (Back-to-Top)

4.3.1 Data

a. Data - Human
b. Data - Synthesis
Nearby in the ranking
Similar items