AI Rank
中文
← Open Source
chujiezheng

chat_templates

Chat Templates for 🤗 HuggingFace Large Language Models.

AI EngineeringPrompt & ScaffoldJinja
Open on GitHub
Momentum
+0stars in 24 hours0.0%
714
Stars
64
Forks
+-1
This week
12
Contributors
Created 2023-11-23 · Updated 2026-10-11 · #14650 today
Top developers
chujiezhengf4strdimaioksha

chujiezheng/chat_templates is ranked 14,650 with 714 stars.

Same-category comparison

chujiezheng/chat_templates is ranked 14,650 among these 8 peers; Picovoice/rhino is ranked 14,647.

Same-category comparison
NameRankMetrics
prompt-security/ps-fuzz14,619
Language:
Python
Stars:
717
Forks:
104
Description:
Make your GenAI Apps Safe & Secure :rocket: Test & harden your system prompt.
MISTLab/Swarm-SLAM14,630
Language:
Shell
Stars:
715
Forks:
82
Description:
Sparse Decentralized Collaborative Simultaneous Localization and Mapping Framework for Multi-Robot Systems.
OpenClaudia/openclaudia-skills14,632
Language:
JavaScript
Stars:
715
Forks:
59
Description:
77 open-source marketing skills for Claude Code, Codex, and other AI coding agents. SEO, content, email, ads, analytics, and growth.
Picovoice/rhino14,647
Language:
Python
Stars:
714
Forks:
97
Description:
On-device Speech-to-Intent engine powered by deep learning.
yvann-ba/ComfyUI_Yvann-Nodes14,657
Language:
Python
Stars:
714
Forks:
30
Description:
Audio Reactivity Nodes for ComfyUI 🔊 Create AI generated audio-driven animations.
mckaywrigley/ai-brainstore14,665
Language:
TypeScript
Stars:
713
Forks:
97
Description:
An experiment concept for an AI brain.
SeanJ1ang/design-judge-skills14,671
Language:
Python
Stars:
712
Forks:
102
Description:
Evidence-driven Agent Skills for design award research, evaluation, award matching, entry writing, and submission readiness.
easylink-ai-open/tiri-agent14,688
Language:
TypeScript
Stars:
711
Forks:
121
Description:
Tiri local agent backend and web shell built on agent-runtime.
Key parameters and specifications
Key parameters and specifications
ParameterValue
LanguageJinja
CategoryAI Engineering
SubcategoryPrompt & Scaffold
Stars714
Forks64
Contributors12
Created date2023-11-23
Updated date2026-10-11
Stars in 1 day0
Stars in 7 days-1
Rank14,650
Previous rank14,503
Rank change-147
7-day stars rank17,684
Previous 7-day stars rank17,638
7-day stars rank change-46
Total stars rank14,660
Previous total stars rank14,638
Total stars rank change-22
CountryChina
README summary

chat_templates **NOTICE: I may not update or maintain this repository actively, as I noticed that recent HF LLMs have usually implemented their chat templates in the file. I am glad about this common practice and it will definitely facilitate out-of-box use of these powerful models. ** This is a repository that includes proper chat templates (or input formats) for instruction-tuned large language models (LLMs), to support 's feature. If you are interested to include more chat templates

, feel free to open a pull request If you find this repo useful, please kindly cite it: Updates **[11/2024]** Added support for Meta's **Llama-3. 2** models **[10/2024]** Added support for IBM's **Granite-3. 0** models **[07/2024]** Added support for Meta's **Llama-3. 1** models **[06/2024]** Added support for Google's **Gemma-2** models **[05/2024]** Added support for Nvidia's **ChatQA** models **[04/2024]** Added support for Microsoft's **Phi-3** models **[04/2024]** Added support for Meta's

**Llama-3** models **[02/2024]** Added support for Google's **Gemma** models **[02/2024]** Added usage explanation for **generation_configs** **[01/2024]** Added support for Alibaba's **Qwen2** models What are Contained in This Repo? contains the jinja files of collected chat templates, which can be directly replaced in the Huggingface tokenizers contains the corresponding json configs used for controlling the ending of response generations. Specially, **the should be directly passed into the method by the argument** Usage Examples **Important NOTE:** As

mentioned in this issue, the should contain **at least one user message**. It is strongly not recommented to pass only the system message, as there may result in unexpected outputs (because the models are not trained in this way). Example 1: Meta-Llama-3-8B-Instruct This example may check if the jinja file is correctly implemented. Expected output: Example 2: Mistral-7B-Instruct-v0. 2 For (also ), it does not natively support the message, so passing the message would

raise error. Expected output: Example 3: vicuna-7b-v1. 5 **NOTE:** In fast-chat, does not add linebreaks between roles' messages. But I found that adding linebreaks leads to a bit better performance (especially for the v1. Also, I found may not work well when given a system message different from its default one. Expected output: Supported Models **NOTE:** The listed models are not inclusive and also include other-sized ones in the same model family granite-3. 0-instruct

Models: , , , Chat template: Generation config: Reference: Llama-3. 2-Instruct Models: , Chat template: Generation config: Reference: Llama-3. 1-Instruct Models: , Chat template: Generation config: Reference: Llama-3-Instruct Models: Chat template: Generation config: Reference: Llama-2-Chat, CodeLlama-Instruct Models: , Chat template: Generation config: Reference: Qwen2-Instruct, Qwen1. 5-Chat Models: , Chat template: Generation config: Reference: Mistral-Instruct Models: , Chat template: Generation config: Reference: Comment: **System message is acceptable** by prepending it before the first user input Phi-3

-Instruct Models: , Chat template: , Generation config: , Reference: **Note:** and adopt different configs Yi-1. 5-Chat, Yi-Chat Models: , Chat

Frequently asked questions

How many stars and forks are recorded?

chujiezheng/chat_templates has 714 stars and 64 forks.

What is the repository rank?

chujiezheng/chat_templates is ranked 14,650.

What language and category are recorded?

chujiezheng/chat_templates has 714 stars, uses Jinja and belongs to AI Engineering.

Which repository is closest in rank?

chujiezheng/chat_templates is compared with Picovoice/rhino, ranked 14,647 with 714 stars.

README

chat_templates

NOTICE: I may not update or maintain this repository actively, as I noticed that recent HF LLMs have usually implemented their chat templates in the tokenizer_config.json file. I am glad about this common practice and it will definitely facilitate out-of-box use of these powerful models.

This is a repository that includes proper chat templates (or input formats) for instruction-tuned large language models (LLMs), to support transformers's chat_template feature. If you are interested to include more chat templates, feel free to open a pull request

If you find this repo useful, please kindly cite it:

@misc{zheng-2024-chat-templates,
  author = {Zheng, Chujie},
  title = {Chat Templates for HuggingFace Large Language Models},
  year = {2024},
  howpublished = {\url{https://github.com/chujiezheng/chat_templates}}
}

Updates

  • [11/2024] Added support for Meta's Llama-3.2 models
  • [10/2024] Added support for IBM's Granite-3.0 models
  • [07/2024] Added support for Meta's Llama-3.1 models
  • [06/2024] Added support for Google's Gemma-2 models
  • [05/2024] Added support for Nvidia's ChatQA models
  • [04/2024] Added support for Microsoft's Phi-3 models
  • [04/2024] Added support for Meta's Llama-3 models
  • [02/2024] Added support for Google's Gemma models
  • [02/2024] Added usage explanation for generation_configs
  • [01/2024] Added support for Alibaba's Qwen2 models

What are Contained in This Repo?

  • chat_templates contains the jinja files of collected chat templates, which can be directly replaced in the Huggingface tokenizers

  • generation_configs contains the corresponding json configs used for controlling the ending of response generations. Specially, the stop_token_ids should be directly passed into the generate method by the eos_token_id argument

Usage Examples

Important NOTE: As mentioned in this issue, the messages should contain at least one user message. It is strongly not recommented to pass only the system message, as there may result in unexpected outputs (because the models are not trained in this way).

Example 1: Meta-Llama-3-8B-Instruct

This example may check if the jinja file is correctly implemented.

from transformers import AutoTokenizer

toker = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct", token="YOUR_OWN_TOKEN")
messages = [
    {'role': 'system', 'content': 'This is a system prompt.'},
    {'role': 'user', 'content': 'This is the first user input.'},
    {'role': 'assistant', 'content': 'This is the first assistant response.'},
    {'role': 'user', 'content': 'This is the second user input.'},
]
print('###### Default (yet Correct) Chat Template ######')
print(toker.apply_chat_template(messages, tokenize=False, add_generation_prompt=True))
print('###### Corrected Chat Template ######')
chat_template = open('./chat_templates/llama-3-instruct.jinja').read()
chat_template = chat_template.replace('    ', '').replace('\n', '')
toker.chat_template = chat_template
print(toker.apply_chat_template(messages, tokenize=False, add_generation_prompt=True))

Expected output:

###### Default (yet Correct) Chat Template ######
<|begin_of_text|><|start_header_id|>system<|end_header_id|>

This is a system prompt.<|eot_id|><|start_header_id|>user<|end_header_id|>

This is the first user input.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

This is the first assistant response.<|eot_id|><|start_header_id|>user<|end_header_id|>

This is the second user input.<|eot_id|><|start_header_id|>assistant<|end_header_id|>


###### Corrected Chat Template ######
<|begin_of_text|><|start_header_id|>system<|end_header_id|>

This is a system prompt.<|eot_id|><|start_header_id|>user<|end_header_id|>

This is the first user input.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

This is the first assistant response.<|eot_id|><|start_header_id|>user<|end_header_id|>

This is the second user input.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Example 2: Mistral-7B-Instruct-v0.2

For mistral-instruct (also gemma-it), it does not natively support the system message, so passing the system message would raise error.

from transformers import AutoTokenizer

toker = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
messages = [
    {'role': 'system', 'content': 'This is a system prompt.'},
    {'role': 'user', 'content': 'This is the first user input.'},
    {'role': 'assistant', 'content': 'This is the first assistant response.'},
    {'role': 'user', 'content': 'This is the second user input.'},
]
print('###### Default (but Improper) Chat Template ######')
# raising error
#print(toker.apply_chat_template(messages, tokenize=False, add_generation_prompt=True))
print('###### Corrected Chat Template ######')
chat_template = open('./chat_templates/mistral-instruct.jinja').read()
chat_template = chat_template.replace('    ', '').replace('\n', '')
toker.chat_template = chat_template
print(toker.apply_chat_template(messages, tokenize=False, add_generation_prompt=True))

Expected output:

###### Default (but Error-Raising) Chat Template ######
jinja2.exceptions.TemplateError: Conversation roles must alternate user/assistant/user/assistant/...
###### Corrected Chat Template ######
[INST] This is a system prompt.

This is the first user input. [/INST] This is the first assistant response. [INST] This is the second user input. [/INST]

Example 3: vicuna-7b-v1.5

NOTE: In fast-chat, vicuna does not add linebreaks between roles' messages. But I found that adding linebreaks leads to a bit better performance (especially for the v1.5 version).

Also, I found vicuna-7/13/33b-v1.3 may not work well when given a system message different from its default one. So I would recommend to use vicuna-7/13b-v1.5 instead.

from transformers import AutoTokenizer

toker = AutoTokenizer.from_pretrained("lmsys/vicuna-7b-v1.5")
messages = [
    {'role': 'system', 'content': 'This is a system prompt.'},
    {'role': 'user', 'content': 'This is the first user input.'},
    {'role': 'assistant', 'content': 'This is the first assistant response.'},
    {'role': 'user', 'content': 'This is the second user input.'},
]
print('###### Default (but Improper) Chat Template ######')
print(toker.apply_chat_template(messages, tokenize=False, add_generation_prompt=True))
print('###### Corrected Chat Template ######')
chat_template = open('./chat_templates/vicuna.jinja').read()
chat_template = chat_template.replace('    ', '').replace('\n', '')
toker.chat_template = chat_template
print(toker.apply_chat_template(messages, tokenize=False, add_generation_prompt=True))

Expected output:

###### Default (but Improper) Chat Template ######
[INST] <>
This is a system prompt.
<>

This is the first user input. [/INST] This is the first assistant response. [INST] This is the second user input. [/INST]
###### Corrected Chat Template ######
This is a system prompt.

USER: This is the first user input.
ASSISTANT: This is the first assistant response.
USER: This is the second user input.
ASSISTANT:

Supported Models

NOTE: The listed models are not inclusive and also include other-sized ones in the same model family

granite-3.0-instruct

Llama-3.2-Instruct

Llama-3.1-Instruct

Llama-3-Instruct

Llama-2-Chat, CodeLlama-Instruct

Qwen2-Instruct, Qwen1.5-Chat

Mistral-Instruct

Phi-3-Instruct

Yi-1.5-Chat, Yi-Chat

gemma-it, gemma-2-it

Llama3-ChatQA-1.5

openchat-3.5, Starling-LM

zephyr

vicuna

Orca-2

falcon-instruct

SOLAR-Instruct

Alpaca

AmberChat

saiga

Star History

Nearby in the ranking
Similar items