Awesome-LLMs-Datasets

Awesome LLMs Datasets

Summarize existing representative LLMs text datasets across five dimensions: Pre-training Corpora, Fine-tuning Instruction Datasets, Preference Datasets, Evaluation Datasets, and Traditional NLP Datasets. (Regular updates)
New dataset sections have been added: Multi-modal Large Language Models (MLLMs) Datasets, Retrieval Augmented Generation (RAG) Datasets. (Gradual updates)
To improve update efficiency and frequency, starting from 2025, only key details of datasets will be updated (e.g., dataset name, paper link, etc.). For more comprehensive information, please refer to the corresponding research papers.

Paper

The paper "Datasets for Large Language Models: A Comprehensive Survey" has been released.（2024/2）

Abstract:

This paper embarks on an exploration into the Large Language Model (LLM) datasets, which play a crucial role in the remarkable advancements of LLMs. The datasets serve as the foundational infrastructure analogous to a root system that sustains and nurtures the development of LLMs. Consequently, examination of these datasets emerges as a critical topic in research. In order to address the current lack of a comprehensive overview and thorough analysis of LLM datasets, and to gain insights into their current status and future trends, this survey consolidates and categorizes the fundamental aspects of LLM datasets from five perspectives: (1) Pre-training Corpora; (2) Instruction Fine-tuning Datasets; (3) Preference Datasets; (4) Evaluation Datasets; (5) Traditional Natural Language Processing (NLP) Datasets. The survey sheds light on the prevailing challenges and points out potential avenues for future investigation. Additionally, a comprehensive review of the existing available dataset resources is also provided, including statistics from 444 datasets, covering 8 language categories and spanning 32 domains. Information from 20 dimensions is incorporated into the dataset statistics. The total data size surveyed surpasses 774.5 TB for pre-training corpora and 700M instances for other datasets. We aim to present the entire landscape of LLM text datasets, serving as a comprehensive reference for researchers in this field and contributing to future studies.

 ![](Fig_1.jpg) 





Fig 1. The overall architecture of the survey. Zoom in for better view

Dataset Information Module

The following is a summary of the dataset information module.

Corpus/Dataset name
Publisher
Release Time
- “X” indicates unknown month.
Size
Public or Not
- “All” indicates full open source;
- “Partial” indicates partially open source;
- “Not” indicates not open source.
License
Language
- “EN” indicates English;
- “ZH” indicates Chinese;
- “AR” indicates Arabic;
- “ES” indicates Spanish;
- “RU” indicates Russian;
- “DE” indicates German;
- “KO” indicates Korean;
- “LT” indicates Lithuanian;
- “FA” indicates Persian/Farsi;
- “PL” indicates Programming Language;
- “Multi” indicates Multilingual, and the number in parentheses indicates the number of languages included.
Construction Method
- “HG” indicates Human Generated Corpus/Dataset;
- “MC” indicates Model Constructed Corpus/Dataset;
- “CI” indicates Collection and Improvement of Existing Corpus/Dataset.
Category
Source
Domain
Instruction Category
Preference Evaluation Method
- “VO” indicates Vote;
- “SO” indicates Sort;
- “SC” indicates Score;
- “-H” indicates Conducted by Humans;
- “-M” indicates Conducted by Models.
Question Type
- “SQ” indicates Subjective Questions;
- “OQ” indicates Objective Questions;
- “Multi” indicates Multiple Question Types.
Evaluation Method
- “CE” indicates Code Evaluation;
- “HE” indicates Human Evaluation;
- “ME” indicates Model Evaluation.
Focus
Numbers of Evaluation Categories/Subcategories
Evaluation Category
Number of Entity Categories (NER Task)
Number of Relationship Categories (RE Task)

Changelog

Click to expand and view hidden content.

（2024/01/17）Create the Awesome-LLMs-Datasets dataset repository.
（2024/02/02）Revise information for some datasets; add Dolma (Pre-training Corpora | General Pre-training Corpora | Multi-category).
（2024/02/15）Add Aya Collection (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | HG & CI & MC); Aya Dataset (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | HG).
（2024/02/22）Add OpenMathInstruct-1 (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Math); FinBen (Evaluation Datasets | Financial).
（2024/04/05）
- Add new dataset sections: (1) Multi-modal Large Language Models (MLLMs) Datasets; (2) Retrieval Augmented Generation (RAG) Datasets.
- Add MMRS-1M (MLLMs Datasets | Instruction Fine-tuning Datasets); VideoChat2-IT (MLLMs Datasets | Instruction Fine-tuning Datasets); InstructDoc (MLLMs Datasets | Instruction Fine-tuning Datasets); ALLaVA-4V Data (MLLMs Datasets | Instruction Fine-tuning Datasets); MVBench (MLLMs Datasets | Evaluation Datasets); OlympiadBench (MLLMs Datasets | Evaluation Datasets); MMMU (MLLMs Datasets | Evaluation Datasets).
- Add CLUE Benchmark Series (Evaluation Datasets | Evaluation Platform); OpenLLM Leaderboard (Evaluation Datasets | Evaluation Platform); OpenCompass (Evaluation Datasets | Evaluation Platform); MTEB Leaderboard (Evaluation Datasets | Evaluation Platform); C-MTEB Leaderboard (Evaluation Datasets | Evaluation Platform).
- Add NAH (Needle-in-a-Haystack) (Evaluation Datasets | Long Text); ToolEyes (Evaluation Datasets | Tool); UHGEval (Evaluation Datasets | Factuality); CLongEval (Evaluation Datasets | Long Text).
- Add MathPile (Pre-training Corpora | Domain-specific Pre-training Corpora | Math); WanJuan-CC (Pre-training Corpora | General Pre-training Corpora | Webpages).
- Add IEPile (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | CI); InstructIE (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | HG).
- Add CRUD-RAG (RAG Datasets); WikiEval (RAG Datasets); RGB (RAG Datasets); RAG-Instruct-Benchmark-Tester (RAG Datasets); ARES (RAG Datasets).
（2024/04/06）
- Add GPQA (Evaluation Datasets | Subject); MGSM (Evaluation Datasets | Multilingual); HaluEval-Wild (Evaluation Datasets | Factuality); CMATH (Evaluation Datasets | Subject); FineMath (Evaluation Datasets | Subject); RealTime QA (Evaluation Datasets | Factuality); WYWEB (Evaluation Datasets | Subject); ChineseFactEval (Evaluation Datasets | Factuality); Counting-Stars (Evaluation Datasets | Long Text).
- Add SlimPajama (Pre-training Corpora | General Pre-training Corpora | Multi-category); MassiveText (Pre-training Corpora | General Pre-training Corpora | Multi-category); MADLAD-400 (Pre-training Corpora | General Pre-training Corpora | Webpages); Minerva (Pre-training Corpora | General Pre-training Corpora | Multi-category); CCAligned (Pre-training Corpora | General Pre-training Corpora | Parallel Corpus); WikiMatrix (Pre-training Corpora | General Pre-training Corpora | Parallel Corpus); OpenWebMath (Pre-training Corpora | Domain-specific Pre-training Corpora | Math).
- Add WebQuestions (Traditional NLP Datasets | Question Answering | Knowledge QA).
- Add ALCE (RAG Datasets).
- Add AlphaFin (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Other); COIG-CQIA (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | HG & CI).
（2024/06/15）
- Add CLUE (Evaluation Datasets | Medical); CHC-Bench (Evaluation Datasets | General); CIF-Bench (Evaluation Datasets | General); ACLUE (Evaluation Datasets | Subject); LeSC (Evaluation Datasets | NLU); AlignBench (Evaluation Datasets | Multitask); SciKnowEval (Evaluation Datasets | Subject).
- Add MAP-CC (Pre-training Corpora | General Pre-training Corpora | Multi-category); FineWeb (Pre-training Corpora | General Pre-training Corpora | Webpages); CCI 2.0 (Pre-training Corpora | General Pre-training Corpora | Webpages).
- Add WildChat (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | MC).
- Add OpenHermesPreferences (Preference Datasets | Sort); huozi_rlhf_data (Preference Datasets | Vote); HelpSteer (Preference Datasets | Score); HelpSteer2 (Preference Datasets | Score).
- Add MMT-Bench (MLLMs Datasets | Evaluation Datasets); mOSCAR (MLLMs Datasets | Pre-training Corpora); MM-NIAH (MLLMs Datasets | Evaluation Datasets).
- Add CRAG (RAG Datasets).
（2024/08/29）
- Add GameBench (Evaluation Datasets | Reasoning); HalluDial (Evaluation Datasets | Factuality); WildBench (Evaluation Datasets | General); DomainEval (Evaluation Datasets | Code); SysBench (Evaluation Datasets | General); KoBEST (Evaluation Datasets | NLU); SarcasmBench (Evaluation Datasets | NLU); C3 Bench (Evaluation Datasets | Subject); TableBench (Evaluation Datasets | Reasoning); ArabLegalEval (Evaluation Datasets | Law).
- Add MultiTrust (MLLMs Datasets | Evaluation Datasets); OBELISC (MLLMs Datasets | Pre-training Corpora); MultiMed (MLLMs Datasets | Evaluation Datasets).
- Add DCLM (Pre-training Corpora | General Pre-training Corpora | Webpages).
- Add Lithuanian-QA-v1 (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | CI & MC); REInstruct (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | HG & CI & MC); KoLLM-Converations (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | CI).
（2024/09/04）
- Add LongWriter-6K (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | CI & MC).
- Add MedTrinity-25M (MLLMs Datasets | Evaluation Datasets); MMIU (MLLMs Datasets | Evaluation Datasets).
- Add Expository-Prose-V1 (Pre-training Corpora | General Pre-training Corpora | Multi-category).
- Add DebateQA (Evaluation Datasets | Knowledge); NeedleBench (Evaluation Datasets | Long Text); ArabicMMLU (Evaluation Datasets | Subject); PersianMMLU (Evaluation Datasets | Subject); TMMLU+ (Evaluation Datasets | Subject).
- Add RAGEval (RAG Datasets); LFRQA (RAG Datasets); MultiHop-RAG (RAG Datasets).
（2024/12/14）
- Add Future-Idea-Generation (Evaluation Datasets | Other); WenMind (Evaluation Datasets | Subject); Chinese SimpleQA (Evaluation Datasets | Knowledge).
- Add MME-RealWorld (MLLMs Datasets | Evaluation Datasets); II-Bench (MLLMs Datasets | Evaluation Datasets); CII-Bench (MLLMs Datasets | Evaluation Datasets); ALM-Bench (MLLMs Datasets | Evaluation Datasets).
- Add MaLA (Pre-training Corpora | General Pre-training Corpora | Multi-category); CCI3.0-HQ (Pre-training Corpora | General Pre-training Corpora | Multi-category); GlotCC (Pre-training Corpora | General Pre-training Corpora | Webpages); ChineseWebText 2.0 (Pre-training Corpora | General Pre-training Corpora | Webpages); ChineseWebText 1.0 (Pre-training Corpora | General Pre-training Corpora | Webpages); SkyPile (Pre-training Corpora | General Pre-training Corpora | Webpages).
- Add ViDoRe (RAG Datasets); M2KR (RAG Datasets); M-BEIR (RAG Datasets); MRAG-Bench (RAG Datasets).
- Add SlimOrca (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | CI & MC); GPTeacher (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | MC); OrcaMathWordProblems (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Math); MathInstruct (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Math); MetaMathQA (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Math); Magicoder-OSS-Instruct-75K (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Code).
- Add UltraInteract (Preference Datasets | Other).
（2025/03/25）Add GneissWeb (Pre-training Corpora | General Pre-training Corpora | Webpages).
（2025/10/11）
- Add MMLU-Redux (Evaluation Datasets | Subject); MMLU-Pro (Evaluation Datasets | Subject); SuperGPQA (Evaluation Datasets | Subject); HumanEval+ (Evaluation Datasets | Code); CRUXEval (Evaluation Datasets | Code); MMMLU (Evaluation Datasets | Multilingual); MCS-Bench (Evaluation Datasets | Other); INCLUDE (Evaluation Datasets | Multilingual); LiveBench (Evaluation Datasets | General); IFEval (Evaluation Datasets | General); WritingBench (Evaluation Datasets | Other); AIME (Evaluation Datasets | Subject); ZebraLogic (Evaluation Datasets | Reasoning); CodeElo (Evaluation Datasets | Code).
- Add Zebra-CoT (MLLMs Datasets | Instruction Fine-tuning Datasets).
- Add Big-Math (Preference Datasets | RL Datasets).

Pre-training Corpora
- General Pre-training Corpora
- Domain-specific Pre-training Corpora
  - Financial
  - Medical
  - Math
  - Other
Instruction Fine-tuning Datasets
- General Instruction Fine-tuning Datasets
- Domain-specific Instruction Fine-tuning Datasets
  - Medical
  - Code
  - Legal
  - Math
  - Education
  - Other
Preference Datasets
- Preference Evaluation Methods
  - Vote
  - Sort
  - Score
  - Other
- RL Datasets
Evaluation Datasets
- General
- Exam
- Subject
- NLU
- Reasoning
- Knowledge
- Long Text
- Tool
- Agent
- Code
- OOD
- Law
- Medical
- Financial
- Social Norms
- Factuality
- Evaluation
- Multitask
- Multilingual
- Other
- Evaluation Platform
Traditional NLP Datasets
Multi-modal Large Language Models (MLLMs) Datasets
Retrieval Augmented Generation (RAG) Datasets

Pre-training Corpora

The pre-training corpora are large collections of text data used during the pre-training process of LLMs.

General Pre-training Corpora

The general pre-training corpora are large-scale datasets composed of extensive text from diverse domains and sources. Their primary characteristic is that the text content is not confined to a single domain, making them more suitable for training general foundational models. Corpora are classified based on data categories.

Dataset information format：

- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website
  - Publisher:
  - Size:
  - License:
  - Source:

Webpages

CC-Stories 2018-6 | Not | EN | CI | Paper | Github | Dataset
- Publisher: Google Brain
- Size: 31 GB
- License: -
- Source: Common Crawl
CC100 2020-7 | All | Multi (100) | CI | Paper | Dataset
- Publisher: Facebook AI
- Size: 2.5 TB
- License: Common Crawl Terms of Use
- Source: Common Crawl
CLUECorpus2020 2020-3 | All | ZH | CI | Paper | Dataset
- Publisher: CLUE Organization
- Size: 100 GB
- License: MIT
- Source: Common Crawl
Common Crawl 2007-X | All | Multi | HG | Website
- Publisher: Common Crawl
- Size: -
- License: Common Crawl Terms of Use
- Source: Web crawler data
CulturaX 2023-9 | All | Multi (167) | CI | Paper | Dataset
- Publisher: University of Oregon et al.
- Size: 27 TB
- License: mC4 & OSCAR license
- Source: mC4, OSCAR
C4 2019-10 | All | EN | CI | Paper | Dataset
- Publisher: Google Research
- Size: 12.68 TB
- License: ODC-BY & Common Crawl Terms of Use
- Source: Common Crawl
mC4 2021-6 | All | Multi (108) | CI | Paper | Dataset
- Publisher: Google Research
- Size: 251 GB
- License: ODC-BY & Common Crawl Terms of Use
- Source: Common Crawl
OSCAR 22.01 2022-1 | All | Multi (151) | CI | Paper | Dataset | Website
- Publisher: Inria
- Size: 8.41 TB
- License: CC0
- Source: Common Crawl
RealNews 2019-5 | All | EN | CI | Paper | Github
- Publisher: University of Washington et al.
- Size: 120 GB
- License: Apache-2.0
- Source: Common Crawl
RedPajama-V2 2023-10 | All | Multi (5) | CI | Github | Dataset | Website
- Publisher: Together Computer
- Size: 30.4 T Tokens
- License: Common Crawl Terms of Use
- Source: Common Crawl, C4, etc.
RefinedWeb 2023-6 | Partial | EN | CI | Paper | Dataset
- Publisher: The Falcon LLM team
- Size: 5000 GB
- License: ODC-BY-1.0
- Source: Common Crawl
WuDaoCorpora-Text 2021-6 | Partial | ZH | HG | Paper | Dataset
- Publisher: BAAI et al.
- Size: 200 GB
- License: MIT
- Source: Chinese webpages
WanJuan-CC 2024-2 | Partial | EN | HG | Paper | Dataset
- Publisher: Shanghai Artifcial Intelligence Laboratory
- Size: 1 T Tokens
- License: CC-BY-4.0
- Source: Common Crawl
MADLAD-400 2023-9 | All | Multi (419) | HG | Paper | Github | Dataset
- Publisher: Google DeepMind et al.
- Size: 2.8 T Tokens
- License: ODL-BY
- Source: Common Crawl
FineWeb 2024-4 | All | EN | CI | Dataset
- Publisher: HuggingFaceFW
- Size: 15 TB Tokens
- License: ODC-BY-1.0
- Source: Common Crawl
CCI 2.0 2024-4 | All | ZH | HG | Dataset1 | Dataset2
- Publisher: BAAI
- Size: 501 GB
- License: CCI Usage Aggrement
- Source: Chinese webpages
DCLM 2024-6 | All | EN | CI | Paper | Github | Dataset | Website
- Publisher: University of Washington et al.
- Size: 279.6 TB
- License: Common Crawl Terms of Use
- Source: Common Crawl
GlotCC 2024-10 | All | Multi (1275) | CI | Paper | Github | Dataset
- Publisher: LMU Munich & Munich Center for Machine Learning et al.
- Size: 2 TB
- License: Common Crawl Terms of Use
- Source: Common Crawl
ChineseWebText 2.0 2024-11 | All | ZH | CI | Paper | Github | Dataset
- Publisher: Chinese Academy of Sciences et al.
- Size: 3.8 TB
- License: Apache-2.0
- Source: MAP-CC, WanJuan, WuDao, etc.
ChineseWebText 1.0 2023-11 | All | ZH | CI | Paper | Github | Dataset
- Publisher: Chinese Academy of Sciences et al.
- Size: 1.42 TB
- License: -
- Source: Common Crawl
SkyPile 2023-10 | Partial | ZH | HG | Paper | Github | Dataset
- Publisher: Kunlun Inc
- Size: 150 B Tokens
- License: Skywork Community License
- Source: Publicly accessible Chinese Internet webpages
GneissWeb 2025-2 | Paper | Dataset
- Publisher: IBM Research et al.

Language Texts

ANC 2003-X | All | EN | HG | Website
- Publisher: The US National Science Foundation et al.
- Size: -
- License: -
- Source: American English texts
BNC 1994-X | All | EN | HG | Website
- Publisher: Oxford University Press et al.
- Size: 4124 Texts
- License: -
- Source: British English texts
News-crawl 2019-1 | All | Multi (59) | HG | Dataset
- Publisher: UKRI et al.
- Size: 110 GB
- License: CC0
- Source: Newspapers

Books

Anna’s Archive 2023-X | All | Multi | HG | Website
- Publisher: Anna
- Size: 586.3 TB
- License: -
- Source: Sci-Hub, Library Genesis, Z-Library, etc.
BookCorpusOpen 2021-5 | All | EN | CI | Paper | Github | Dataset
- Publisher: Jack Bandy et al.
- Size: 17,868 Books
- License: Smashwords Terms of Service
- Source: Toronto Book Corpus
PG-19 2019-11 | All | EN | HG | Paper | Github | Dataset
- Publisher: DeepMind
- Size: 11.74 GB
- License: Apache-2.0
- Source: Project Gutenberg
Project Gutenberg 1971-X | All | Multi | HG | Website
- Publisher: Ibiblio et al.
- Size: -
- License: The Project Gutenberg
- Source: Ebook data
Smashwords 2008-X | All | Multi | HG | Website
- Publisher: Draft2Digital et al.
- Size: -
- License: Smashwords Terms of Service
- Source: Ebook data
Toronto Book Corpus 2015-6 | Not | EN | HG | Paper | Website
- Publisher: University of Toronto et al.
- Size: 11,038 Books
- License: MIT & Smashwords Terms of Service
- Source: Smashwords

Academic Materials

arXiv 1991-X | All | EN | HG | Website
- Publisher: Paul Ginsparg et al.
- Size: -
- License: Terms of Use for arXiv APIs
- Source: arXiv preprint
S2ORC 2020-6 | All | EN | CI | Paper | Github
- Publisher: AI2 et al.
- Size: 81.1 MB
- License: ODC-BY-1.0
- Source: MAG, arXiv, PubMed, etc.

Code

BIGQUERY 2022-3 | Not | PL | CI | Paper | Github
- Publisher: Salesforce Research
- Size: 341.1 GB
- License: Apache-2.0
- Source: BigQuery
Github 2008-4 | All | PL | HG | Website
- Publisher: Microsoft
- Size: -
- License: -
- Source: Various code projects
phi-1 2023-6 | Not | EN & PL | HG & MC | Paper | Dataset
- Publisher: Microsoft Research
- Size: 7 B Tokens
- License: CC-BY-NC-SA-3.0
- Source: The Stack, StackOverflow, GPT-3.5 Generation
The Stack 2022-11 | All | PL (358) | HG | Paper | Dataset
- Publisher: ServiceNow Research et al.
- Size: 6 TB
- License: The Terms of the Original Licenses
- Source: Permissively-licensed source code files

Parallel Corpus

MTP 2023-9 | All | EN & ZH | HG & CI | Dataset
- Publisher: BAAI
- Size: 1.3 TB
- License: BAAI Data Usage Protocol
- Source: Chinese-English parallel text pairs on the web
MultiUN 2010-5 | All | Multi (7) | HG | Paper | Website
- Publisher: German Research Center for Artificial Intelligence (DFKI) GmbH
- Size: 4353 MB
- License: -
- Source: United Nations documents
ParaCrawl 2020-7 | All | Multi (42) | HG | Paper | Website
- Publisher: Prompsit et al.
- Size: 59996 Files
- License: CC0
- Source: Web crawler data
UNCorpus v1.0 2016-5 | All | Multi (6) | HG | Paper | Website
- Publisher: United Nations et al.
- Size: 799276 Files
- License: -
- Source: United Nations documents
CCAligned 2020-11 | All | Multi (138) | HG | Paper | Dataset
- Publisher: Facebook AI et al.
- Size: 392 M URL pairs
- License: -
- Source: Common Crawl
WikiMatrix 2021-4 | All | Multi (85) | HG | Paper | Github | Dataset
- Publisher: Facebook AI et al.
- Size: 134 M parallel sentences
- License: CC-BY-SA
- Source: Wikipedia

Social Media

OpenWebText 2019-4 | All | EN | HG | Website
- Publisher: Brown University
- Size: 38 GB
- License: CC0
- Source: Reddit
Pushshift Reddit 2020-1 | All | EN | CI | Paper | Website
- Publisher: Pushshift.io et al.
- Size: 2 TB
- License: -
- Source: Reddit
Reddit 2005-6 | All | EN | HG | Website
- Publisher: Condé Nast Digital et al.
- Size: -
- License: -
- Source: Social media posts
StackExchange 2008-9 | All | EN | HG | Dataset | Website
- Publisher: Stack Exchange
- Size: -
- License: CC-BY-SA-4.0
- Source: Community question and answer data
WebText 2019-2 | Partial | EN | HG | Paper | Github | Dataset
- Publisher: OpenAI
- Size: 40 GB
- License: MIT
- Source: Reddit
Zhihu 2011-1 | All | ZH | HG | Website
- Publisher: Beijing Zhizhe Tianxia Technology Co., Ltd
- Size: -
- License: Zhihu User Agreement
- Source: Social media posts

Encyclopedia

Baidu baike 2008-4 | All | ZH | HG | Website
- Publisher: Baidu
- Size: -
- License: Baidu baike User Agreement
- Source: Encyclopedic content data
TigerBot-wiki 2023-5 | All | ZH | HG | Paper | Github | Dataset
- Publisher: TigerBot
- Size: 205 MB
- License: Apache-2.0
- Source: Baidu baike
Wikipedia 2001-1 | All | Multi | HG | Dataset | Website
- Publisher: Wikimedia Foundation
- Size: -
- License: CC-BY-SA-3.0 & GFDL
- Source: Encyclopedic content data

Multi-category

ArabicText 2022 2022-12 | All | AR | HG & CI | Dataset
- Publisher: BAAI et al.
- Size: 201.9 GB
- License: CC-BY-SA-4.0
- Source: ArabicWeb, OSCAR, CC100, etc.
MNBVC 2023-1 | All | ZH | HG & CI | Github | Dataset
- Publisher: Liwu Community
- Size: 20811 GB
- License: MIT
- Source: Chinese books, webpages, theses, etc.
RedPajama-V1 2023-4 | All | Multi | HG & CI | Github | Dataset
- Publisher: Together Computer
- Size: 1.2 T Tokens
- License: -
- Source: Common Crawl, Github, books, etc.
ROOTS 2023-3 | Partial | Multi (59) | HG & CI | Paper | Dataset
- Publisher: Hugging Face et al.
- Size: 1.61 TB
- License: BLOOM Open-RAIL-M
- Source: OSCAR, Github, etc.
The Pile 2021-1 | All | EN | HG & CI | Paper | Github | Dataset
- Publisher: EleutherAI
- Size: 825.18 GB
- License: MIT
- Source: Books, arXiv, Github, etc.
TigerBot_pretrain_en 2023-5 | Partial | EN | CI | Paper | Github | Dataset
- Publisher: TigerBot
- Size: 51 GB
- License: Apache-2.0
- Source: English books, webpages, en-wiki, etc
TigerBot_pretrain_zh 2023-5 | Partial | ZH | HG | Paper | Github | Dataset
- Publisher: TigerBot
- Size: 55 GB
- License: Apache-2.0
- Source: Chinese books, webpages, zh-wiki, etc.
WanJuanText-1.0 2023-8 | All | ZH | HG | Paper | Github | Dataset
- Publisher: Shanghai AI Laboratory
- Size: 1094 GB
- License: CC-BY-4.0
- Source: Webpages, Encyclopedia, Books, etc
Dolma 2024-1 | All | EN | HG & CI | Paper | Github | Dataset
- Publisher: AI2 et al.
- Size: 11519 GB
- License: MR Agreement
- Source: Project Gutenberg, C4, Reddit, etc.
SlimPajama 2023-6 | All | EN | HG & CI | Github | Dataset | Website
- Publisher: Cerebras et al.
- Size: 627 B Tokens
- License: -
- Source: Common Crawl, C4, Github, etc.
MassiveText 2021-12 | Not | Multi | HG & CI | Paper
- Publisher: Google DeepMind
- Size: 10.5 TB
- License: -
- Source: MassiveWeb, C4, books, etc.
Minerva 2022-6 | Not | EN | HG | Paper
- Publisher: Google Research
- Size: 38.5 B Tokens
- License: -
- Source: arXiv, Webpages, etc.
MAP-CC 2024-4 | All | ZH | HG | Paper | Github | Dataset | Website
- Publisher: Multimodal Art Projection Research Community et al.
- Size: 840.48 B Token
- License: CC-BY-NC-ND-4.0
- Source: Chinese Common Crawl, Chinese Encyclopedias, Chinese Books etc.
Expository-Prose-V1 2024-8 | All | EN | HG & CI | Paper | Github | Dataset
- Publisher: Pints.ai Labs
- Size: 56 B Tokens
- License: MIT
- Source: ArXiv, Wikipedia, Gutenberg, etc.
MaLA 2024-9 | All | Multi (939) | HG & CI | Paper | Dataset
- Publisher: University of Helsinki et al.
- Size: 74 B Tokens
- License: ODC-BY
- Source: CC100, CulturaX, MADLAD-400 etc.
CCI3.0-HQ 2024-9 | All | ZH | HG | Paper | Dataset
- Publisher: BAAI
- Size: 500 GB
- License: BAAI Data Usage Protocol
- Source: News, Social media, Blogs, etc.

Domain-specific Pre-training Corpora

Domain-specific pre-training corpora are LLM datasets customized for specific fields or topics. The type of corpus is typically employed in the incremental pre-training phase of LLMs. Corpora are classified based on data domains.

Dataset information format：

- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website
  - Publisher:
  - Size:
  - License:
  - Source:
  - Category:
  - Domain:

Financial

BBT-FinCorpus 2023-2 | Partial | ZH | HG | Paper | Github | Website
- Publisher: Fudan University et al.
- Size: 256 GB
- License: -
- Source: Company announcements, research reports, financial
- Category: Multi
- Domain: Finance
FinCorpus 2023-9 | All | ZH | HG | Paper | Github | Dataset
- Publisher: Du Xiaoman
- Size: 60.36 GB
- License: Apache-2.0
- Source: Company announcements, financial news, financial exam questions
- Category: Multi
- Domain: Finance
FinGLM 2023-7 | All | ZH | HG | Github
- Publisher: Knowledge Atlas et al.
- Size: 69 GB
- License: Apache-2.0
- Source: Annual Reports of Listed Companies
- Category: Language Texts
- Domain: Finance
TigerBot-earning 2023-5 | All | ZH | HG | Paper | Github | Dataset
- Publisher: TigerBot
- Size: 488 MB
- License: Apache-2.0
- Source: Financial reports
- Category: Language Texts
- Domain: Finance
TigerBot-research 2023-5 | All | ZH | HG | Paper | Github | Dataset
- Publisher: TigerBot
- Size: 696 MB
- License: Apache-2.0
- Source: Research reports
- Category: Language Texts
- Domain: Finance

Medical

Medical-pt 2023-5 | All | ZH | CI | Github | Dataset
- Publisher: Ming Xu
- Size: 632.78 MB
- License: Apache-2.0
- Source: Medical encyclopedia data, medical textbooks
- Category: Multi
- Domain: Medical
PubMed Central 2000-2 | All | EN | HG | Website
- Publisher: NCBI
- Size: -
- License: PMC Copyright Notice
- Source: Biomedical scientific literature
- Category: Academic Materials
- Domain: Medical

Math

Proof-Pile-2 2023-10 | All | EN | HG & CI | Paper | Github | Dataset | Website
- Publisher: Princeton University et al.
- Size: 55 B Tokens
- License: -
- Source: ArXiv, OpenWebMath, AlgebraicStack
- Category: Multi
- Domain: Mathematics
MathPile 2023-12 | All | EN | HG | Paper | Github | Dataset
- Publisher: Shanghai Jiao Tong University et al.
- Size: 9.5 B Tokens
- License: CC-BY-NC-SA-4.0
- Source: Textbooks, Wikipedia, ProofWiki, CommonCrawl, StackExchange, arXiv
- Category: Multi
- Domain: Mathematics
OpenWebMath 2023-10 | All | EN | HG | Paper | Github | Dataset
- Publisher: University of Toronto et al.
- Size: 14.7 B Tokens
- License: ODC-BY-1.0
- Source: Common Crawl
- Category: Webpages
- Domain: Mathematics

Other

TigerBot-law 2023-5 | All | ZH | HG | Paper | Github | Dataset
- Publisher: TigerBot
- Size: 29.9 MB
- License: Apache-2.0
- Source: Legal clauses
- Category: Language Texts
- Domain: Law
TransGPT-pt 2023-7 | All | ZH | HG | Github | Dataset
- Publisher: Beijing Jiaotong University
- Size: 35.8 MB
- License: Apache-2.0
- Source: Technology documents, engineering construction information, statistical data, etc.
- Category: Multi
- Domain: Transportation

Instruction Fine-tuning Datasets

The instruction fine-tuning datasets consists of a series of text pairs comprising “instruction inputs” and “answer outputs.” “Instruction inputs” represent requests made by humans to the model. There are various types of instructions, such as classification, summarization, paraphrasing, etc. “Answer outputs” are the responses generated by the model following the instruction and aligning with human expectations.

General Instruction Fine-tuning Datasets

General instruction fine-tuning datasets contain one or more instruction categories with no domain restrictions, primarily aiming to enhance the instruction-following capability of LLMs in general tasks. Datasets are classified based on construction methods.

Dataset information format：

- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website
  - Publisher:
  - Size:
  - License:
  - Source:
  - Instruction Category:

Human Generated Datasets (HG)

databricks-dolly-15K 2023-4 | All | EN | HG | Dataset | Website
- Publisher: Databricks
- Size: 15011 instances
- License: CC-BY-SA-3.0
- Source: Manually generated based on different instruction categories
- Instruction Category: Multi
InstructionWild_v2 2023-6 | All | EN & ZH | HG | Github
- Publisher: National University of Singapore
- Size: 110K instances
- License: -
- Source: Collected on the web
- Instruction Category: Multi
LCCC 2020-8 | All | ZH | HG | Paper | Github
- Publisher: Tsinghua University et al.
- Size: 12M instances
- License: MIT
- Source: Crawl user interactions on social media
- Instruction Category: Multi
OASST1 2023-4 | All | Multi (35) | HG | Paper | Github | Dataset
- Publisher: OpenAssistant
- Size: 161443 instances
- License: Apache-2.0
- Source: Generated and annotated by humans
- Instruction Category: Multi
OL-CC 2023-6 | All | ZH | HG | Dataset
- Publisher: BAAI
- Size: 11655 instances
- License: Apache-2.0
- Source: Generated and annotated by humans
- Instruction Category: Multi
Zhihu-KOL 2023-3 | All | ZH | HG | Github | Dataset
- Publisher: wangrui6
- Size: 1006218 instances
- License: MIT
- Source: Crawl from Zhihu
- Instruction Category: Multi
Aya Dataset 2024-2 | All | Multi (65) | HG | Paper | Dataset | Website
- Publisher: Cohere For AI Community et al.
- Size: 204K instances
- License: Apache-2.0
- Source: Manually collected and annotated via the Aya Annotation Platform
- Instruction Category: Multi
InstructIE 2023-5 | All | EN & ZH | HG | Paper | Github | Dataset
- Publisher: Zhejiang University et al.
- Size: 371700 instances
- License: MIT
- Source: Baidu baike, Wikipedia
- Instruction Category: Extraction

Model Constructed Datasets (MC)

Alpaca_data 2023-3 | All | EN | MC | Github
- Publisher: Stanford Alpaca
- Size: 52K instances
- License: Apache-2.0
- Source: Generated by Text-Davinci-003 with Aplaca_data prompts
- Instruction Category: Multi
BELLE_Generated_Chat 2023-5 | All | ZH | MC | Github | Dataset
- Publisher: BELLE
- Size: 396004 instances
- License: GPL-3.0
- Source: Generated by ChatGPT
- Instruction Category: Generation
BELLE_Multiturn_Chat 2023-5 | All | ZH | MC | Github | Dataset
- Publisher: BELLE
- Size: 831036 instances
- License: GPL-3.0
- Source: Generated by ChatGPT
- Instruction Category: Multi
BELLE_train_0.5M_CN 2023-4 | All | ZH | MC | Github | Dataset
- Publisher: BELLE
- Size: 519255 instances
- License: GPL-3.0
- Source: Generated by Text-Davinci-003
- Instruction Category: Multi
BELLE_train_1M_CN 2023-4 | All | ZH | MC | Github | Dataset
- Publisher: BELLE
- Size: 917424 instances
- License: GPL-3.0
- Source: Generated by Text-Davinci-003
- Instruction Category: Multi
BELLE_train_2M_CN 2023-5 | All | ZH | MC | Github | Dataset
- Publisher: BELLE
- Size: 2M instances
- License: GPL-3.0
- Source: Generated by ChatGPT
- Instruction Category: Multi
BELLE_train_3.5M_CN 2023-5 | All | ZH | MC | Github | Dataset
- Publisher: BELLE
- Size: 3606402 instances
- License: GPL-3.0
- Source: Generated by ChatGPT
- Instruction Category: Multi
CAMEL 2023-3 | All | Multi & PL | MC | Paper | Github | Dataset | Website
- Publisher: KAUST
- Size: 1659328 instances
- License: CC-BY-NC-4.0
- Source: Dialogue generated by two GPT-3.5-Turbo agents
- Instruction Category: Multi
Chatgpt_corpus 2023-6 | All | ZH | MC | Github
- Publisher: PlexPt
- Size: 3270K instances
- License: GPL-3.0
- Source: Generated by GPT-3.5-Turbo
- Instruction Category: Multi
InstructionWild_v1 2023-3 | All | EN & ZH | MC | Github
- Publisher: National University of Singapore
- Size: 104K instances
- License: -
- Source: Generated by OpenAI API
- Instruction Category: Multi
LMSYS-Chat-1M 2023-9 | All | Multi | MC | Paper | Dataset
- Publisher: UC Berkeley et al.
- Size: 1M instances
- License: LMSYS-Chat-1M license
- Source: Generated by multiple LLMs
- Instruction Category: Multi
MOSS_002_sft_data 2023-4 | All | EN & ZH | MC | Github | Dataset
- Publisher: Fudan University
- Size: 1161137 instances
- License: CC-BY-NC-4.0
- Source: Generated by Text-Davinci-003
- Instruction Category: Multi
MOSS_003_sft_data 2023-4 | All | EN & ZH | MC | Github | Dataset
- Publisher: Fudan University
- Size: 1074551 instances
- License: CC-BY-NC-4.0
- Source: Conversation data from MOSS-002 and generated by GPT-3.5-Turbo
- Instruction Category: Multi
MOSS_003_sft_plugin_data 2023-4 | Partial | EN & ZH | MC | Github | Dataset
- Publisher: Fudan University
- Size: 300K instances
- License: CC-BY-NC-4.0
- Source: Generated by plugins and LLMs
- Instruction Category: Multi
OpenChat 2023-7 | All | EN | MC | Paper | Github | Dataset
- Publisher: Tsinghua University et al.
- Size: 70K instances
- License: MIT
- Source: ShareGPT
- Instruction Category: Multi
RedGPT-Dataset-V1-CN 2023-4 | Partial | ZH | MC | Github
- Publisher: DA-southampton
- Size: 50K instances
- License: Apache-2.0
- Source: Generated by LLMs
- Instruction Category: Multi
Self-Instruct 2022-12 | All | EN | MC | Paper | Github
- Publisher: University of Washington et al.
- Size: 52445 instances
- License: Apache-2.0
- Source: Generated by GPT-3
- Instruction Category: Multi
ShareChat 2023-4 | All | Multi | MC | Website
- Publisher: Sharechat
- Size: 90K instances
- License: CC0
- Source: ShareGPT
- Instruction Category: Multi
ShareGPT-Chinese-English-90k 2023-7 | All | EN & ZH | MC | Github | Dataset
- Publisher: shareAI
- Size: 90K instances
- License: Apache-2.0
- Source: ShareGPT
- Instruction Category: Multi
ShareGPT90K 2023-4 | All | EN | MC | Dataset
- Publisher: RyokoAI
- Size: 90K instances
- License: CC0
- Source: ShareGPT
- Instruction Category: Multi
UltraChat 2023-5 | All | EN | MC | Paper | Github
- Publisher: Tsinghua University
- Size: 1468352 instances
- License: CC-BY-NC-4.0
- Source: Dialogue generated by two ChatGPT agents
- Instruction Category: Multi
Unnatural Instructions 2022-12 | All | EN | MC | Paper | Github
- Publisher: Tel Aviv University et al.
- Size: 240670 instances
- License: MIT
- Source: Generated by LLMs
- Instruction Category: Multi
WebGLM-QA 2023-6 | All | EN | MC | Paper | Github | Dataset
- Publisher: Tsinghua University et al.
- Size: 44979 instances
- License: Apache-2.0
- Source: Construct WebGLM-QA via LLM in-context bootstrapping
- Instruction Category: Open QA
Wizard_evol_instruct_196K 2023-6 | All | EN | MC | Paper | Github | Dataset
- Publisher: Microsoft et al.
- Size: 196K instances
- License: -
- Source: Evolve instructions through the Evol-Instruct method
- Instruction Category: Multi
Wizard_evol_instruct_70K 2023-5 | All | EN | MC | Paper | Github | Dataset
- Publisher: Microsoft et al.
- Size: 70K instances
- License: -
- Source: Evolve instructions through the Evol-Instruct method
- Instruction Category: Multi
WildChat 2024-5 | Partial | Multi (68) | MC | Paper | Dataset
- Publisher: Cornell University et al.
- Size: 1039785 instances
- License: AI2 ImpACT license
- Source: Conversations between users and ChatGPT, GPT-4
- Instruction Category: Multi
GPTeacher 2023-4 | All | EN | MC | Github | Dataset
- Publisher: teknium
- Size: 89260 instances
- License: MIT
- Source: Generated by GPT-4
- Instruction Category: Multi

Collection and Improvement of Existing Datasets (CI)

CrossFit 2021-4 | All | EN | CI | Paper | Github
- Publisher: University of Southern California
- Size: 269 datasets
- License: -
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
DialogStudio 2023-7 | All | EN | CI | Paper | Github | Dataset
- Publisher: Salesforce AI et al.
- Size: 87 datasets
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
Dynosaur 2023-5 | All | EN | CI | Paper | Github | Dataset | Website
- Publisher: UCLA et al.
- Size: 801900 instances
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
Flan-mini 2023-7 | All | EN | CI | Paper | Github | Dataset
- Publisher: Singapore University of Technology and Design
- Size: 1.34M instances
- License: CC
- Source: Collection and improvement of various instruction fine-tuning datasets
- Instruction Category: Multi
Flan 2021 2021-9 | All | Multi | CI | Paper | Github
- Publisher: Google Research
- Size: 62 datasets
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
Flan 2022 2023-1 | Partial | Multi | CI | Paper | Github | Dataset
- Publisher: Google Research
- Size: 1836 datasets
- License: Apache-2.0
- Source: Collection and improvement of various instruction fine-tuning datasets
- Instruction Category: Multi
InstructDial 2022-5 | All | EN | CI | Paper | Github
- Publisher: Carnegie Mellon University
- Size: 59 datasets
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
NATURAL INSTRUCTIONS 2021-4 | All | EN | CI | Paper | Github | Dataset
- Publisher: Allen Institute for AI et al.
- Size: 61 datasets
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
OIG 2023-3 | All | EN | CI | Dataset
- Publisher: LAION
- Size: 3878622 instances
- License: Apache-2.0
- Source: Collection and improvement of various datasets
- Instruction Category: Multi
Open-Platypus 2023-8 | All | EN | CI | Paper | Github | Dataset | Website
- Publisher: Boston University
- Size: 24926 instances
- License: -
- Source: Collection and improvement of various datasets
- Instruction Category: Multi
OPT-IML Bench 2022-12 | Not | Multi | CI | Paper | Github
- Publisher: Meta AI
- Size: 2000 datasets
- License: MIT
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
PromptSource 2022-2 | All | EN | CI | Paper | Github
- Publisher: Brown University et al.
- Size: 176 datasets
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
SUPER-NATURAL INSTRUCTIONS 2022-4 | All | Multi | CI | Paper | Github
- Publisher: Univ. of Washington et al.
- Size: 1616 datasets
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
T0 2021-10 | All | EN | CI | Paper | Dataset1 | Dataset2
- Publisher: Hugging Face et al.
- Size: 62 datasets
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
UnifiedSKG 2022-3 | All | EN | CI | Paper | Github
- Publisher: The University of Hong Kong et al.
- Size: 21 datasets
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
xP3 2022-11 | All | Multi (46) | CI | Paper | Github
- Publisher: Hugging Face et al.
- Size: 82 datasets
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
IEPile 2024-2 | All | EN & ZH | CI | Paper | Github | Dataset
- Publisher: Zhejiang University et al.
- Size: 33 datasets
- License: CC-BY-NC-SA-4.0
- Source: Collection and improvement of various IE datasets
- Instruction Category: Extraction
KOLLM-Conversations 2024-3 | All | KO | CI | Dataset
- Publisher: davidkim205
- Size: 1122566 instances
- License: Apache-2.0
- Source: Collection and improvement of Korean datasets
- Instruction Category: Multi

HG & CI

Firefly 2023-4 | All | ZH | HG & CI | Github | Dataset
- Publisher: YeungNLP
- Size: 1649399 instances
- License: -
- Source: Collect Chinese NLP datasets and manually generate data related to Chinese culture
- Instruction Category: Multi
LIMA-sft 2023-5 | All | EN | HG & CI | Paper | Dataset
- Publisher: Meta AI et al.
- Size: 1330 instances
- License: CC-BY-NC-SA
- Source: Manually select from various types of data
- Instruction Category: Multi
COIG-CQIA 2024-3 | All | ZH | HG & CI | Paper | Dataset
- Publisher: Shenzhen Institute of Advanced Technology et al.
- Size: 48375 instances
- License: -
- Source: Q&A communities, Wikipedia, examinations, existing NLP datasets
- Instruction Category: Multi

HG & MC

InstructGPT-sft 2022-3 | Not | EN | HG & MC | Paper
- Publisher: OpenAI
- Size: 14378 instances
- License: -
- Source: Platform Q&A data and manual labeling
- Instruction Category: Multi

CI & MC

Alpaca_GPT4_data 2023-4 | All | EN | CI & MC | Paper | Github
- Publisher: Microsoft Research
- Size: 52K instances
- License: Apache-2.0
- Source: Generated by GPT-4 with Aplaca_data prompts
- Instruction Category: Multi
Alpaca_GPT4_data_zh 2023-4 | All | ZH | CI & MC | Github | Dataset
- Publisher: Microsoft Research
- Size: 52K instances
- License: Apache-2.0
- Source: Generated by GPT-4 with Alpaca_data prompts translated into Chinese by ChatGPT
- Instruction Category: Multi
Bactrain-X 2023-5 | All | Multi (52) | CI & MC | Paper | Github | Dataset
- Publisher: MBZUAI
- Size: 3484884 instances
- License: CC-BY-NC-4.0
- Source: Generated by GPT-3.5-Turbo with Aplaca_data and databricks-dolly-15K prompts translated into 51 languages by Google Translate API
- Instruction Category: Multi
Baize 2023-3 | Partial | EN | CI & MC | Paper | Github | Dataset
- Publisher: University of California et al.
- Size: 210311 instances
- License: GPL-3.0
- Source: Sample seeds from specific datasets to create multi-turn dialogues using ChatGPT
- Instruction Category: Multi
GPT4All 2023-3 | All | EN | CI & MC | Paper | Github | Dataset
- Publisher: nomic-ai
- Size: 739259 instances
- License: MIT
- Source: Generated by GPT-3.5-Turbo with other datasets’ prompts
- Instruction Category: Multi
GuanacoDataset 2023-3 | All | Multi | CI & MC | Dataset | Website
- Publisher: JosephusCheung
- Size: 534530 instances
- License: GPL-3.0
- Source: Expand upon the initial 52K dataset from the Alpaca model
- Instruction Category: Multi
LaMini-LM 2023-4 | All | EN | CI & MC | Paper | Github | Dataset
- Publisher: Monash University et al.
- Size: 2585615 instances
- License: CC-BY-NC-4.0
- Source: Generated by ChatGPT with synthetic and existing prompts
- Instruction Category: Multi
LogiCoT 2023-5 | All | EN & ZH | CI & MC | Paper | Github | Dataset
- Publisher: Westlake University et al.
- Size: 604840 instances
- License: CC-BY-NC-ND-4.0
- Source: Expand the datasets using GPT-4
- Instruction Category: Reasoning
LongForm 2023-4 | All | EN | CI & MC | Paper | Github | Dataset
- Publisher: LMU Munich et al.
- Size: 27739 instances
- License: MIT
- Source: Select documents from existing corpora and generating prompts for the documents using LLMs
- Instruction Category: Multi
Luotuo-QA-B 2023-5 | All | EN & ZH | CI & MC | Github | Dataset
- Publisher: Luotuo
- Size: 157320 instances
- License: Apache-2.0 & CC0
- Source: Use LLMs to generate Q&A pairs on CSL, arXiv, and CNN-DM datasets
- Instruction Category: Multi
OpenOrca 2023-6 | All | Multi | CI & MC | Paper | Dataset
- Publisher: Microsoft Researc
- Size: 4233923 instances
- License: MIT
- Source: Expand upon the Flan 2022 dataset using GPT-3.5-Turbo and GPT-4
- Instruction Category: Multi
Wizard_evol_instruct_zh 2023-5 | All | ZH | CI & MC | Github | Dataset
- Publisher: Central China Normal University et al.
- Size: 70K instances
- License: CC-BY-4.0
- Source: Generated by GPT with Wizard_evol_instruct prompts translated into Chinese
- Instruction Category: Multi
Lithuanian-QA-v1 2024-8 | All | LT | CI & MC | Paper | Dataset
- Publisher: Neurotechnology
- Size: 13848 instances
- License: CC-BY-4.0
- Source: Use ChatGPT to generate Q&A pairs on Wikipedia corpus
- Instruction Category: Multi
LongWriter-6K 2024-8 | All | EN & ZH | CI & MC | Paper | Github | Dataset
- Publisher: Tsinghua University et al.
- Size: 6000 instances
- License: Apache-2.0
- Source: Generated by GPT-4o with open-source datasets’ prompts
- Instruction Category: Multi
SlimOrca 2023-10 | All | EN | CI & MC | Dataset
- Publisher: Open-Orca
- Size: 517,982 instances
- License: MIT
- Source: Generated by GPT-4 and from OpenOrca data
- Instruction Category: Multi

HG & CI & MC

COIG 2023-4 | All | ZH | HG & CI & MC | Paper | Github | Dataset
- Publisher: BAAI
- Size: 191191 instances
- License: Apache-2.0
- Source: Translated instructions, Leetcode, Chinese exams, etc.
- Instruction Category: Multi
HC3 2023-1 | All | EN & ZH | HG & CI & MC | Paper | Github | Dataset1 | Dataset2
- Publisher: SimpleAI
- Size: 37175 instances
- License: CC-BY-SA-4.0
- Source: Human-Q&A pairs and ChatGPT-Q&A pairs from Q&A platforms, encyclopedias, etc.
- Instruction Category: Multi
Phoenix-sft-data-v1 2023-5 | All | Multi | HG & CI & MC | Paper | Github | Dataset
- Publisher: The Chinese University of Hong Kong et al.
- Size: 464510 instances
- License: CC-BY-4.0
- Source: Collected multi-lingual instructions, post-translated multi-lingual instructions, self-generated user-centered multi-lingual instructions
- Instruction Category: Multi
TigerBot_sft_en 2023-5 | Partial | EN | HG & CI & MC | Paper | Github | Dataset
- Publisher: TigerBot
- Size: 677117 instances
- License: Apache-2.0
- Source: Self-instruct, human-labeling, open-source data cleaning
- Instruction Category: Multi
TigerBot_sft_zh 2023-5 | Partial | ZH | HG & CI & MC | Paper | Github | Dataset
- Publisher: TigerBot
- Size: 530705 instances
- License: Apache-2.0
- Source: Self-instruct, human-labeling, open-source data cleaning
- Instruction Category: Multi
Aya Collection 2024-2 | All | Multi (114) | HG & CI & MC | Paper | Dataset | Website
- Publisher: Cohere For AI Community et al.
- Size: 513M instances
- License: Apache-2.0
- Source: Templated data, Translated data and Aya Dataset
- Instruction Category: Multi
REInstruct 2024-8 | Not | EN | HG & CI & MC | Paper | Github
- Publisher: Chinese Information Processing Laboratory et al.
- Size: 35K instances
- License: -
- Source: Automatically constructing instruction data from the C4 corpus using a small amount of manually annotated seed instruction data
- Instruction Category: Multi

Domain-specific Instruction Fine-tuning Datasets

The domain-specific instruction fine-tuning datasets are constructed for a particular domain by formulating instructions that encapsulate knowledge and task types closely related to that domain.

Dataset information format：

- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website
  - Publisher:
  - Size:
  - License:
  - Source:
  - Instruction Category:
  - Domain:

Medical

ChatDoctor 2023-3 | All | EN | HG & MC | Paper | Github | Dataset
- Publisher: University of Texas Southwestern Medical Center et al.
- Size: 115K instances
- License: Apache-2.0
- Source: Real conversations between doctors and patients & Generated by ChatGPT
- Instruction Category: Multi
- Domain: Medical
ChatMed_Consult_Dataset 2023-5 | All | ZH | MC | Github | Dataset
- Publisher: michael-wzhu
- Size: 549326 instances
- License: CC-BY-NC-4.0
- Source: Generated by GPT-3.5-Turbo
- Instruction Category: Multi
- Domain: Medical
CMtMedQA 2023-8 | All | ZH | HG | Paper | Github | Dataset
- Publisher: Zhengzhou University
- Size: 68023 instances
- License: MIT
- Source: Real conversations between doctors and patients
- Instruction Category: Multi
- Domain: Medical
DISC-Med-SFT 2023-8 | All | ZH | HG & CI | Paper | Github | Dataset | Website
- Publisher: Fudan University et al.
- Size: 464898 instances
- License: Apache-2.0
- Source: Open source datasets & Manually selected data
- Instruction Category: Multi
- Domain: Medical
HuatuoGPT-sft-data-v1 2023-5 | All | ZH | HG & MC | Paper | Github | Dataset
- Publisher: The Chinese University of Hong Kong et al.
- Size: 226042 instances
- License: Apache-2.0
- Source: Real conversations between doctors and patients & Generated by ChatGPT
- Instruction Category: Multi
- Domain: Medical
Huatuo-26M 2023-5 | Partial | ZH | CI | Paper | Github
- Publisher: The Chinese University of Hong Kong et al.
- Size: 26504088 instances
- License: Apache-2.0
- Source: Collection and improvement of various datasets
- Instruction Category: Multi
- Domain: Medical
MedDialog 2020-4 | All | EN & ZH | HG | Paper | Github
- Publisher: UC San Diego
- Size: 3.66M instances
- License: -
- Source: Real conversations between doctors and patients
- Instruction Category: Multi
- Domain: Medical
Medical Meadow 2023-4 | All | EN | HG & CI | Paper | Github | Dataset
- Publisher: University Hospital Aachen et al.
- Size: 160076 instances
- License: GPL-3.0
- Source: Crawl data from the Internet & Collection and improvement of various NLP datasets
- Instruction Category: Multi
- Domain: Medical
Medical-sft 2023-5 | All | EN & ZH | CI | Github | Dataset
- Publisher: Ming Xu
- Size: 2.07M instances
- License: Apache-2.0
- Source: Collection and improvement of various NLP datasets
- Instruction Category: Multi
- Domain: Medical
QiZhenGPT-sft-20k 2023-5 | Partial | ZH | CI | Github | Dataset
- Publisher: Zhejiang University
- Size: 20K instances
- License: GPL-3.0
- Source: Collection and improvement of various datasets
- Instruction Category: Multi
- Domain: Medical
ShenNong_TCM_Dataset 2023-6 | All | ZH | MC | Github | Dataset
- Publisher: michael-wzhu
- Size: 112565 instances
- License: Apache-2.0
- Source: Generated by ChatGPT
- Instruction Category: Multi
- Domain: Medical

Code

Code_Alpaca_20K 2023-3 | All | EN & PL | MC | Github | Dataset
- Publisher: Sahil Chaudhary
- Size: 20K instances
- License: Apache-2.0
- Source: Generated by Text-Davinci-003
- Instruction Category: Code
- Domain: Code
CodeContest 2022-3 | All | EN & PL | CI | Paper | Github
- Publisher: DeepMind
- Size: 13610 instances
- License: Apache-2.0
- Source: Collection and improvement of various datasets
- Instruction Category: Code
- Domain: Code
CommitPackFT 2023-8 | All | EN & PL (277) | HG | Paper | Github | Dataset
- Publisher: Bigcode
- Size: 702062 instances
- License: MIT
- Source: GitHub Action dump
- Instruction Category: Code
- Domain: Code
ToolAlpaca 2023-6 | All | EN & PL | HG & MC | Paper | Github
- Publisher: Chinese Information Processing Laboratory et al.
- Size: 3928 instances
- License: Apache-2.0
- Source: Manually filter APIs & Generated by ChatGPT
- Instruction Category: Code
- Domain: Code
ToolBench 2023-7 | All | EN & PL | HG & MC | Paper | Github
- Publisher: Tsinghua University et al.
- Size: 126486 instances
- License: Apache-2.0
- Source: Manually filter APIs & Generated by ChatGPT
- Instruction Category: Code
- Domain: Code
Magicoder-OSS-Instruct-75K 2023-12 | All | EN | CI & MC | Paper | Github | Dataset
- Publisher: University of Illinois at Urbana-Champaign et al.
- Size: 75197 instances
- License: MIT
- Source: Generated by prompting LLMs with seed code snippets from open-source repositories to create diverse coding problems and solutions
- Instruction Category: Code
- Domain: Code

Legal

DISC-Law-SFT 2023-9 | Partial | ZH | HG & CI & MC | Paper | Github | Website
- Publisher: Fudan University et al.
- Size: 403K instances
- License: Apache-2.0
- Source: Open source datasets & Legal-related Text Content & Generated by GPT-3.5-Turbo
- Instruction Category: Multi
- Domain: Law
HanFei 1.0 2023-5 | All | ZH | - | Github | Dataset
- Publisher: Chinese Academy of Sciences et al.
- Size: 255K instances
- License: Apache-2.0
- Source: Filter legal-related data according to rules
- Instruction Category: Multi
- Domain: Law
LawGPT_zh 2023-5 | Partial | ZH | CI & MC | Github | Dataset
- Publisher: Shanghai Jiao Tong University
- Size: 200K instances
- License: -
- Source: Real conversations & Generated by ChatGPT
- Instruction Category: Multi
- Domain: Law
Lawyer LLaMA_sft 2023-5 | Partial | ZH | CI & MC | Paper | Github | Dataset
- Publisher: Peking Universit
- Size: 21476 instances
- License: Apache-2.0
- Source: Generated by ChatGPT with other datasets’ prompts
- Instruction Category: Multi
- Domain: Law

Math

BELLE_School_Math 2023-5 | All | ZH | MC | Github | Dataset
- Publisher: BELLE
- Size: 248481 instances
- License: GPL-3.0
- Source: Generated by ChatGPT
- Instruction Category: Math
- Domain: Math
Goat 2023-5 | All | EN | HG | Paper | Github | Dataset
- Publisher: National University of Singapore
- Size: 1746300 instances
- License: Apache-2.0
- Source: Artificially synthesized data
- Instruction Category: Math
- Domain: Math
MWP 2021-9 | All | EN & ZH | CI | Paper | Github | Dataset
- Publisher: Xihua University et al.
- Size: 251598 instances
- License: MIT
- Source: Collection and improvement of various datasets
- Instruction Category: Math
- Domain: Math
OpenMathInstruct-1 2024-2 | All | EN | CI & MC | Paper | Github | Dataset
- Publisher: NVIDIA
- Size: 1.8M instances
- License: NVIDIA License
- Source: GSM8K and MATH datasets (original questions); Generated using Mixtral-8×7B model
- Instruction Category: Math
- Domain: Math
OrcaMathWordProblems 2024-2 | All | EN | CI & MC | Paper | Dataset
- Publisher: Microsoft Research
- Size: 200035 instances
- License: MIT
- Source: Generated by GPT-4 Turbo
- Instruction Category: Math
- Domain: Math
MathInstruct 2023-9 | All | EN | CI & MC | Paper | Github | Dataset | Website
- Publisher: University of Waterloo et al.
- Size: 262039 instances
- License: MIT
- Source: GSM8K, MATH, etc. and use GPT-4 to generate CoT and PoT reasoning processes
- Instruction Category: Math
- Domain: Math
MetaMathQA 2023-9 | All | EN | CI & MC | Paper | Github | Dataset | Website
- Publisher: University of Cambridge et al.
- Size: 395000 instances
- License: MIT
- Source: Augmented from the GSM8K and MATH datasets using LLMs
- Instruction Category: Math
- Domain: Math

Education

Child_chat_data 2023-8 | All | ZH | HG & MC | Github
- Publisher: Harbin Institute of Technology et al.
- Size: 5000 instances
- License: -
- Source: Real conversations & Generated by GPT-3.5-Turbo
- Instruction Category: Multi
- Domain: Education
Educhat-sft-002-data-osm 2023-7 | All | EN & ZH | CI | Paper | Github | Dataset
- Publisher: East China Normal University et al.
- Size: 4279419 instances
- License: CC-BY-NC-4.0
- Source: Collection and improvement of various datasets
- Instruction Category: Multi
- Domain: Education
TaoLi_data 2023-X | All | ZH | HG & CI | Github | Dataset
- Publisher: Beijing Language and Culture University et al.
- Size: 88080 instances
- License: Apache-2.0
- Source: Collection and improvement of various datasets & Manually extract dictionary data
- Instruction Category: Multi
- Domain: Education

Other

DISC-Fin-SFT 2023-10 | Partial | ZH | HG & CI & MC | Paper | Github | Website
- Publisher: Fudan University et al.
- Size: 246K instances
- License: Apache-2.0
- Source: Open source datasets & Manually collect financial data & ChatGPT assistance
- Instruction Category: Multi
- Domain: Financial
AlphaFin 2024-3 | All | EN & ZH | HG & CI & MC | Paper | Github | Dataset
- Publisher: South China University of Technology et al.
- Size: 167362 instances
- License: Apache-2.0
- Source: Traditional research datasets, real-time financial data, handwritten CoT data
- Instruction Category: Multi
- Domain: Financial
GeoSignal 2023-6 | Partial | EN | HG & CI & MC | Paper | Github | Dataset
- Publisher: Shanghai Jiao Tong University et al.
- Size: 22627272 instances
- License: Apache-2.0
- Source: Open source datasets & Geoscience-related Text Content & Generated by GPT-4
- Instruction Category: Multi
- Domain: Geoscience
MeChat 2023-4 | All | ZH | CI & MC | Paper | Github | Dataset
- Publisher: Zhejiang University et al.
- Size: 56K instances
- License: CC0-1.0
- Source: Based on PsyQA dataset with the proposed SMILE method
- Instruction Category: Multi
- Domain: Mental Health
Mol-Instructions 2023-6 | All | EN | HG & CI & MC | Paper | Github | Dataset
- Publisher: Zhejiang University et al.
- Size: 2043586 instances
- License: CC-BY-4.0
- Source: Molecule-oriented, Protein-oriented, Biomolecular text instructions
- Instruction Category: Multi
- Domain: Biology
Owl-Instruction 2023-9 | All | EN & ZH | HG & MC | Paper | Github
- Publisher: Beihang University et al.
- Size: 17858 instances
- License: -
- Source: Generated by GPT-4 & Manual verification
- Instruction Category: Multi
- Domain: IT
PROSOCIALDIALOG 2022-5 | All | EN | HG & MC | Paper | Dataset
- Publisher: Allenai
- Size: 165681 instances
- License: CC-BY-4.0
- Source: Generated by humans with GPT-3 created prompts
- Instruction Category: Social Norms
- Domain: Social Norms
TransGPT-sft 2023-7 | All | ZH | HG | Github | Dataset
- Publisher: Beijing Jiaotong University
- Size: 58057 instances
- License: Apache-2.0
- Source: Manually collect traffic-related data
- Instruction Category: Multi
- Domain: Transportation

Preference Datasets

Preference datasets are collections of instructions that provide preference evaluations for multiple responses to the same instruction input.

Preference Evaluation Methods

The preference evaluation methods for preference datasets can be categorized into voting, sorting, scoring, and other methods. Datasets are classified based on preference evaluation methods.

Dataset information format：

- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website
  - Publisher:
  - Size:
  - License:
  - Domain:
  - Instruction Category: 
  - Preference Evaluation Method: 
  - Source:

Vote

Chatbot_arena_conversations 2023-6 | All | Multi | HG & MC | Paper | Dataset
- Publisher: UC Berkeley et al.
- Size: 33000 instances
- License: CC-BY-4.0 & CC-BY-NC-4.0
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: VO-H
- Source: Generated by twenty LLMs & Manual judgment
hh-rlhf 2022-4 | All | EN | HG & MC | Paper1 | Paper2 | Github | Dataset
- Publisher: Anthropic
- Size: 169352 instances
- License: MIT
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: VO-H
- Source: Generated by LLMs & Manual judgment
MT-Bench_human_judgments 2023-6 | All | EN | HG & MC | Paper | Github | Dataset | Website
- Publisher: UC Berkeley et al.
- Size: 3.3K instances
- License: CC-BY-4.0
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: VO-H
- Source: Generated by LLMs & Manual judgment
PKU-SafeRLHF 2023-7 | Partial | EN | HG & CI & MC | Paper | Github | Dataset
- Publisher: Peking University
- Size: 361903 instances
- License: CC-BY-NC-4.0
- Domain: Social Norms
- Instruction Category: Social Norms
- Preference Evaluation Method: VO-H
- Source: Generated by LLMs & Manual judgment
SHP 2021-10 | All | EN | HG | Paper | Github | Dataset
- Publisher: Stanford
- Size: 385563 instances
- License: -
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: VO-H
- Source: Reddit data & Manual judgment
Zhihu_rlhf_3k 2023-4 | All | ZH | HG | Dataset
- Publisher: Liyucheng
- Size: 3460 instances
- License: CC-BY-2.0
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: VO-H
- Source: Zhihu data & Manual judgment
Summarize_from_Feedback 2020-9 | All | EN | HG & CI | Paper | Dataset
- Publisher: OpenAI
- Size: 193841 instances
- License: -
- Domain: News
- Instruction Category: Multi
- Preference Evaluation Method: VO-H & SC-H
- Source: Open source datasets & Manual judgment and scoring
CValues 2023-7 | All | ZH | MC | Paper | Github | Dataset
- Publisher: Alibaba Group
- Size: 145K instances
- License: Apache-2.0
- Domain: Social Norms
- Instruction Category: Social Norms
- Preference Evaluation Method: VO-M
- Source: Generated by LLMs & Evaluation by the reward model
huozi_rlhf_data 2024-2 | All | ZH | HG & MC | Github | Dataset
- Publisher: Huozi-Team
- Size: 16918 instances
- License: Apache-2.0
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: VO-H
- Source: Generated by Huozi model & Manual judgment

Sort

OASST1_pairwise_rlhf_reward 2023-5 | All | Multi | CI | Dataset
- Publisher: Tasksource
- Size: 18918 instances
- License: Apache-2.0
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: SO-H
- Source: OASST1 datasets & Manual sorting

Score

Stack-Exchange-Preferences 2021-12 | All | EN | HG | Paper | Dataset
- Publisher: Anthropic
- Size: 10807695 instances
- License: CC-BY-SA-4.0
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: SC-H
- Source: Stackexchange data & Manual scoring
WebGPT 2021-12 | All | EN | HG & CI | Paper | Dataset
- Publisher: OpenAI
- Size: 19578 instances
- License: -
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: SC-H
- Source: Open source datasets & Manual scoring
Alpaca_comparison_data 2023-3 | All | EN | MC | Github
- Publisher: Stanford Alpaca
- Size: 51K instances
- License: Apache-2.0
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: SC-M
- Source: Generated by three LLMs & GPT-4 scoring
Stable_Alignment 2023-5 | All | EN | MC | Paper | Github
- Publisher: Dartmouth College et al.
- Size: 169K instances
- License: Apache-2.0
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: SC-M
- Source: Generated by LLMs & Model scoring
UltraFeedback 2023-10 | All | EN | CI & MC | Paper | Github | Dataset
- Publisher: Tsinghua University et al.
- Size: 63967 instances
- License: MIT
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: SC-M
- Source: Generated by seventeen LLMs & Model scoring
OpenHermesPreferences 2024-3 | All | EN | CI & MC | Dataset
- Publisher: Argilla et al.
- Size: 989490 instances
- License: -
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: SO-M
- Source: OpenHermes-2.5 dataset & Model sorting
HelpSteer 2023-11 | All | EN | HG & CI & MC | Paper | Dataset
- Publisher: NVIDIA
- Size: 37120 instances
- License: CC-BY-4.0
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: SC-H
- Source: Generated by LLMs & Manual judgment
HelpSteer2 2024-6 | All | EN | HG & CI & MC | Paper | Github | Dataset
- Publisher: NVIDIA
- Size: 21362 instances
- License: CC-BY-4.0
- Domain: General
- Instruction Category: Multi
- Preference Evaluation Method: SC-H
- Source: Generated by LLMs & Manual judgment

Other

Medical-rlhf 2023-5 | All | ZH | CI & MC | Github | Dataset
- Publisher: Ming Xu
- Size: 4K instances
- License: Apache-2.0
- Domain: Medical
- Instruction Category: Multi
- Preference Evaluation Method: Other
- Source: Response_chosen comes from the doctor's response & Response_rejected comes from the model's response
PRM800K 2023-5 | All | EN | HG & CI & MC | Paper | Github
- Publisher: OpenAI
- Size: 800K instances
- License: MIT
- Domain: Math
- Instruction Category: Math
- Preference Evaluation Method: Other
- Source: Generated by LLMs & Mathematical reasoning steps are determined manually
UltraInteract 2024-4 | All | EN | CI & MC | Paper | Dataset
- Publisher: Tsinghua University et al.
- Size: 219000 instances
- License: MIT
- Domain: General
- Instruction Category: Multi (Code, Math, Reasoning)
- Preference Evaluation Method: Other
- Source: CodeContest, MATH, etc.

RL Datasets

Big-Math 2025-2 | Paper | Github | Dataset
- Publisher: SynthLabs et al.

Evaluation Datasets

Evaluation datasets are a carefully curated and annotated set of data samples used to assess the performance of LLMs across various tasks.Datasets are classified based on evaluation domains.

Dataset information format：

- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website
  - Publisher:
  - Size:
  - License:
  - Question Type: 
  - Evaluation Method: 
  - Focus: 
  - Numbers of Evaluation Categories/Subcategories: 
  - Evaluation Category:

General

AlpacaEval 2023-5 | All | EN | CI & MC | Paper | Github | Dataset | Website
- Publisher: Stanford et al.
- Size: 805 instances
- License: Apache-2.0
- Question Type: SQ
- Evaluation Method: ME
- Focus: The performance on open-ended question answering
- Numbers of Evaluation Categories/Subcategories: 1/-
- Evaluation Category: Open-ended question answering
BayLing-80 2023-6 | All | EN & ZH | HG & CI | Paper | Github | Dataset
- Publisher: Chinese Academy of Sciences
- Size: 320 instances
- License: GPL-3.0
- Question Type: SQ
- Evaluation Method: ME
- Focus: Chinese-English language proficiency and multimodal interaction skills
- Numbers of Evaluation Categories/Subcategories: 9/-
- Evaluation Category: Writing, Roleplay, Common-sense, Fermi, Counterfactual, Coding, Math, Generic, Knowledge
BELLE_eval 2023-4 | All | ZH | HG & MC | Paper | Github
- Publisher: BELLE
- Size: 1000 instances
- License: Apache-2.0
- Question Type: SQ
- Evaluation Method: ME
- Focus: The performance of Chinese language models in following instructions
- Numbers of Evaluation Categories/Subcategories: 9/-
- Evaluation Category: Extract, Closed qa, Rewrite, Summarization, Generation, Classification, Brainstorming, Open qa, Others
CELLO 2023-9 | All | EN | HG | Paper | Github
- Publisher: Fudan University et al.
- Size: 523 instances
- License: -
- Question Type: SQ
- Evaluation Method: CE
- Focus: The ability of LLMs to understand complex instructions
- Numbers of Evaluation Categories/Subcategories: 2/10
- Evaluation Category: Complex task description, Complex input
MT-Bench 2023-6 | All | EN | HG | Paper | Github | Website
- Publisher: UC Berkeley et al.
- Size: 80 instances
- License: Apache-2.0
- Question Type: SQ
- Evaluation Method: ME
- Focus: The performance on open-ended question answering
- Numbers of Evaluation Categories/Subcategories: 8/-
- Evaluation Category: Writing, Roleplay, Reasoning, Math, Coding, Extraction, STEM, Humanities
SuperCLUE 2023-7 | Not | ZH | HG & MC | Paper | Github | Website1 | Website2
- Publisher: CLUE et al.
- Size: 3754 instances
- License: -
- Question Type: Multi
- Evaluation Method: HE & CE
- Focus: The performance in a Chinese context
- Numbers of Evaluation Categories/Subcategories: 2/-
- Evaluation Category: Open multi-turn open questions, OPT objective questions
Vicuna Evaluation 2023-3 | All | EN | HG | Github | Dataset | Website
- Publisher: LMSYS ORG
- Size: 80 instances
- License: Apache-2.0
- Question Type: SQ
- Evaluation Method: ME
- Focus: The performance on open-ended question answering
- Numbers of Evaluation Categories/Subcategories: 9/-
- Evaluation Category: Generic, Knowledge, Roleplay, Common-sense, Fermi, Counterfactual, Coding, Math, Writing
CHC-Bench 2024-4 | All | ZH | HG & CI | Paper | Github | Dataset | Website
- Publisher: Multimodal Art Projection Research Community et al.
- Size: 214 instances
- License: Apache-2.0
- Question Type: Multi
- Evaluation Method: ME
- Focus: Hard-case Chinese instructions understanding and following
- Numbers of Evaluation Categories/Subcategories: 8/-
- Evaluation Category: Writing, Humanity, Science, Role-playing, Reading Comprehension, Math, Hard Cases, Coding
CIF-Bench 2024-2 | Partial | ZH | HG & CI | Paper | Github | Website
- Publisher: University of Manchester et al.
- Size: 15K instances
- License: -
- Question Type: SQ
- Evaluation Method: CE & ME
- Focus: Evaluate the zero-shot generalizability of LLMs to the Chinese language
- Numbers of Evaluation Categories/Subcategories: 10/150
- Evaluation Category: Chinese culture, Classification, Code, Commonsense, Creative NLG, Evaluation, Grammar, Linguistic, Motion detection, NER
WildBench 2024-6 | All | EN | HG & CI | Paper | Github | Dataset | Website
- Publisher: Allen Institute for AI et al.
- Size: 1024 instances
- License: AI2 ImpACT License
- Question Type: SQ
- Evaluation Method: ME
- Focus: An automated evaluation framework designed to benchmark LLMs using challenging, real-world user queries.
- Numbers of Evaluation Categories/Subcategories: 11/-
- Evaluation Category: Information seeking, Coding & Debugging, Creative writing, Reasoning, Planning, Math, Editing, Data analysis, Role playing, Brainstorming, Advice seeking
SysBench 2024-8 | All | EN | HG | Paper | Github | Dataset
- Publisher: Peking University et al.
- Size: 500 instances
- License: -
- Question Type: SQ
- Evaluation Method: ME
- Focus: Systematically analyze system message following ability
- Numbers of Evaluation Categories/Subcategories: 3/-
- Evaluation Category: Constraint complexity, Instruction misalignment, Multi-turn stability
LiveBench 2024-6 | All | EN | HG & CI | Paper | Github | Dataset | Website
- Publisher: Abacus.AI et al.
- Size: 1000 instances (regularly updated)
- License: Apache-2.0
- Question Type: Multi
- Evaluation Method: CE
- Focus: Contamination-limited evaluation of LLMs across math, coding, reasoning, language, instruction following, and data analysis
- Numbers of Evaluation Categories/Subcategories: 6/18
- Evaluation Category: Math, Coding, Reasoning, Language, Instruction Following, Data Analysis
IFEval 2023-11 | All | Multi | HG | Paper | Github | Dataset
- Publisher: Google et al.
- Size: 541 instances
- License: Apache-2.0
- Question Type: SQ
- Evaluation Method: ME
- Focus: Instruction-following ability of LLMs via verifiable instructions
- Numbers of Evaluation Categories/Subcategories: 9/25
- Evaluation Category: Keywords, Language, Length Constraints, Detectable Content, Detectable Format, Combination, Change Cases, Start with/End with, Punctuation

Exam

AGIEval 2023-4 | All | EN & ZH | HG & CI | Paper | Github | Dataset
- Publisher: Microsoft
- Size: 8062 instances
- License: MIT
- Question Type: OQ
- Evaluation Method: CE
- Focus: Human-centric standardized exams
- Numbers of Evaluation Categories/Subcategories: 7/20
- Evaluation Category: Gaokao, SAT, JEC, LSAT, LogiQA, AQuA-RAT, Math
GAOKAO-Bench 2023-5 | All | ZH | HG | Paper | Github
- Publisher: Fudan University et al.
- Size: 2811 instances
- License: Apache-2.0
- Question Type: Multi
- Evaluation Method: HE & CE
- Focus: Chinese Gaokao examination
- Numbers of Evaluation Categories/Subcategories: 10/-
- Evaluation Category: Chinese, Mathematics (2 categories), English, Physics, Chemistry, Biology, Politics, History, Geography
M3Exam 2023-6 | All | Multi (9) | HG | Paper | Github
- Publisher: Alibaba Group et al.
- Size: 12317 instances
- License: -
- Question Type: OQ
- Evaluation Method: CE
- Focus: The comprehensive abilities in a multilingual and multilevel context using real human exam questions
- Numbers of Evaluation Categories/Subcategories: 3/-
- Evaluation Category: Low, Mid, High

Subject

ARB 2023-7 | All | EN | CI | Paper | Github
- Publisher: DuckAI et al.
- Size: 1207 instances
- License: MIT
- Question Type: Multi
- Evaluation Method: HE & ME
- Focus: Advanced reasoning problems in multiple fields
- Numbers of Evaluation Categories/Subcategories: 5/-
- Evaluation Category: Mathematics, Physics, Law, MCAT(Reading), MCAT(Science)
C-CLUE 2021-8 | All | ZH | HG | Github | Website
- Publisher: Tianjin University
- Size: -
- License: CC-BY-SA-4.0
- Question Type: SQ
- Evaluation Method: CE
- Focus: Classical Chinese language understanding
- Numbers of Evaluation Categories/Subcategories: 2/-
- Evaluation Category: Named entity recognition, Relation extraction
C-Eval 2023-5 | All | ZH | HG & MC | Paper | Github | Dataset | Website
- Publisher: Shanghai Jiao Tong University
- Size: 13948 instances
- License: CC-BY-NC-SA-4.0
- Question Type: OQ
- Evaluation Method: CE
- Focus: The advanced knowledge and reasoning abilities in a Chinese context
- Numbers of Evaluation Categories/Subcategories: 4/52
- Evaluation Category: STEM, Social Science, Humanity, Other
CG-Eval 2023-8 | All | ZH | HG | Paper | Github | Dataset | Website
- Publisher: LanguageX AI Lab et al.
- Size: 11000 instances
- License: CC-BY-SA-4.0
- Question Type: SQ
- Evaluation Method: CE
- Focus: The generation capabilities of LLMs across various academic disciplines
- Numbers of Evaluation Categories/Subcategories: 6/55
- Evaluation Category: Science and engineering, Humanities and social sciences, Mathematical calculations, Medical practitioner qualification Examination, Judicial Examination, Certfied public accountant examination
LLMEVAL-3 2023-9 | Not | ZH | HG | Github | Website
- Publisher: Fudan University et al.
- Size: 200K instances
- License: -
- Question Type: SQ
- Evaluation Method: ME
- Focus: Subject-specific knowledge capability
- Numbers of Evaluation Categories/Subcategories: 13/-
- Evaluation Category: Philosophy, Economics, Law, Education, Literature, History, Science, Engineering, Agriculture, Medicine, Military science, Management, Fine arts
MMCU 2023-4 | All | ZH | HG | Paper | Github
- Publisher: LanguageX AI Lab
- Size: 11845 instances
- License: -
- Question Type: OQ
- Evaluation Method: CE
- Focus: Multidisciplinary abilities
- Numbers of Evaluation Categories/Subcategories: 4/25
- Evaluation Category: Medicine, Law, Psychology, Education
MMLU 2020-9 | All | EN | HG | Paper | Github
- Publisher: UC Berkeley et al.
- Size: 15908 instances
- License: MIT
- Question Type: OQ
- Evaluation Method: CE
- Focus: Knowledge in academic and professional domains
- Numbers of Evaluation Categories/Subcategories: 4/57
- Evaluation Category: Humanities, Social science, STEM, Other
M3KE 2023-5 | All | ZH | HG | Paper | Github | Dataset
- Publisher: Tianjin University et al.
- Size: 20477 instances
- License: Apache-2.0
- Question Type: OQ
- Evaluation Method: CE
- Focus: Multidisciplinary abilities
- Numbers of Evaluation Categories/Subcategories: 4/71
- Evaluation Category: Arts & Humanities, Social sciences, Natural sciences, Other
SCIBENCH 2023-7 | All | EN | HG | Paper | Github
- Publisher: University of California et al.
- Size: 695 instances
- License: MIT
- Question Type: SQ
- Evaluation Method: CE
- Focus: The performance in university-level science and engineering domains
- Numbers of Evaluation Categories/Subcategories: 3/10
- Evaluation Category: Physics, Chemistry, Math
ScienceQA 2022-9 | All | EN | HG | Paper | Github | Website
- Publisher: University of California et al.
- Size: 21208 instances
- License: CC-BY-NC-SA-4.0
- Question Type: OQ
- Evaluation Method: CE
- Focus: Science question-answering ability
- Numbers of Evaluation Categories/Subcategories: 3/26
- Evaluation Category: Natural science, Social science, Language science
TheoremQA 2023-5 | All | EN | HG | Paper | Github | Dataset
- Publisher: University of Waterloo et al.
- Size: 800 instances
- License: MIT
- Question Type: SQ
- Evaluation Method: CE
- Focus: Science subject question-answering ability
- Numbers of Evaluation Categories/Subcategories: 4/39
- Evaluation Category: Mathematics, Physics, Finance, CS & EE
XiezhiBenchmark 2023-6 | All | EN & ZH | HG & MC | Paper | Github
- Publisher: Fudan University et al.
- Size: 249587 instances
- License: CC-BY-NC-SA-4.0
- Question Type: OQ
- Evaluation Method: CE
- Focus: Multidisciplinary abilities
- Numbers of Evaluation Categories/Subcategories: 13/516
- Evaluation Category: Medicine, Literature, Economics, Agronomy, Science, Jurisprudence, History, Art studies, Philosophy, Pedagogy, Military science, Management, Engineering
CMMLU 2023-6 | All | ZH | HG | Paper | Github | Dataset
- Publisher: MBZUAI
- Size: 11528 instances
- License: CC-BY-NC-4.0
- Question Type: OQ
- Evaluation Method: CE
- Focus: The knowledge and reasoning capabilities within the Chinese context
- Numbers of Evaluation Categories/Subcategories: 5/67
- Evaluation Category: Social science, STEM, Humanities, China specific, Other
GPQA 2023-11 | All | EN | HG | Paper | Github | Dataset
- Publisher: New York University et al.
- Size: 448 instances
- License: CC-BY-4.0
- Question Type: OQ
- Evaluation Method: CE
- Focus: The disciplinary knowledge in the fields of biology, physics, and chemistry
- Numbers of Evaluation Categories/Subcategories: 3/16
- Evaluation Category: Biology, Physics, Chemistry
CMATH 2023-6 | All | ZH | HG | Paper | Github | Dataset
- Publisher: Xiaomi AI Lab
- Size: 1698 instances
- License: CC-BY-4.0
- Question Type: SQ
- Evaluation Method: CE
- Focus: Elementary school math word problems
- Numbers of Evaluation Categories/Subcategories: 6/-
- Evaluation Category: Grades 1 to

Paper

Dataset Information Module

Changelog

Table of Contents

Pre-training Corpora

General Pre-training Corpora

Webpages

Language Texts

Books

Academic Materials

Code

Parallel Corpus

Social Media

Encyclopedia

Multi-category

Domain-specific Pre-training Corpora

Financial

Medical

Math

Other

Instruction Fine-tuning Datasets

General Instruction Fine-tuning Datasets

Human Generated Datasets (HG)

Model Constructed Datasets (MC)

Collection and Improvement of Existing Datasets (CI)

HG & CI

HG & MC

CI & MC

HG & CI & MC

Domain-specific Instruction Fine-tuning Datasets

Medical

Code

Legal

Math

Education

Other

Preference Datasets

Preference Evaluation Methods

Vote

Sort

Score

Other

RL Datasets

Evaluation Datasets

General

Exam

Subject