AI Rank
EN
← 开源
Troyanovsky

Local-LLM-Comparison-Colab-UI

Compare the performance of different LLM that can be deployed locally on consumer hardware. Run yourself with Colab WebUI.

ListsModel collectionsJupyter Notebook
在 GitHub 打开
增长势头
+024 小时新增 Star0.0%
1.11k
Star
155
Fork
+2
本周
3
贡献者
创建于 2023-05-07 · 更新于 2026-10-14 · 今日第 11571 名
主要开发者
TroyanovskyklipskiRonbalt

Troyanovsky/Local-LLM-Comparison-Colab-UI 排名第 11,571，有 1,107 个星标。

同类别比较

Troyanovsky/Local-LLM-Comparison-Colab-UI 排名第 11,571，此处有 8 个同类条目；Kim-Hammar/awesome-rl-for-cybersecurity 排名第 11,586。

同类别比较
名称排名指标
leomaurodesenv/game-datasets11,465
星标数:
1,134
分叉数:
94
简介:
:video_game: A curated list of awesome game datasets, and tools to artificial intelligence in games
offchan42/machine-learning-curriculum11,468
星标数:
1,133
分叉数:
253
简介:
:computer: Learn to make machines learn so that you don't have to struggle to program them; The ultimate list
Shujian2015/FreeML11,470
星标数:
1,132
分叉数:
511
简介:
A List of Data Science/Machine Learning Resources (Mostly Free)
TSCenter/awesome-time-series-papers11,477
星标数:
1,130
分叉数:
54
简介:
An Awesome List of the latest time series papers and code from top AI venues.
Kim-Hammar/awesome-rl-for-cybersecurity11,586
星标数:
1,103
分叉数:
146
简介:
A curated list of resources dedicated to reinforcement learning applied to cyber security.
Chen-Cai-OSU/awesome-equivariant-network11,601
星标数:
1,100
分叉数:
108
简介:
Paper list for equivariant neural network
qualisero/awesome-pi-agent11,606
语言:
JavaScript
星标数:
1,100
分叉数:
66
简介:
Awesome list of add-ons, hooks, tools, skills, and resources for the pi coding agent (pi-mono).
abubakarsiddik31/claude-skills-collection11,608
星标数:
1,099
分叉数:
204
简介:
A curated collection of official and community-built Claude Skills – extend Anthropic's Claude with powerful, modular capabilities for productivity, creativity, coding, and more.
关键参数与规格
关键参数与规格
参数值
语言Jupyter Notebook
类别Lists
子类别Model collections
关键词LocalAI
星标数1,107
分叉数155
贡献者3
创建日期2023-05-07
更新日期2026-10-14
1 天星标数0
7 天星标数2
排名11,571
上次排名5,997
排名变化-5,574
7 天星标排名8,228
上次 7 天星标排名8,297
7 天星标排名变化69
总星标排名10,304
上次总星标排名10,293
总星标排名变化-11
README 摘录

Local LLM Comparison & Colab Links (WIP) (Update Nov. 27, 2023) The original goal of the repo was to compare some smaller models (7B and 13B) that can be run on consumer hardware so every model had a score for a set of questions from GPT-4. But I realized that as there are many more capable models appearing, the evaluation and comparison process may not suffice. Therefore, I'm only putting Colab WebUI links for the newer models and you can

try them out yourselves with a few clicks - after all, the effectiveness of a language model relies heavily on its suitability for your specific use case. By trying out the models firsthand, you can assess their performance and determine which one best fits your needs. Newer Models without Scores: These models can be run on consumer hardware and are generally good (from Reddit suggestions and my own experience). Try them out yourselves (tap the 'Open in Colab' button)! 5-Mistral

-7B-GGUF | | 2023-11-27 | Roleplay good, not censored | | | zephyr-7B-beta-GGUF | | 2023-11-27 | Roleplay okay, not censored | | | dolphin-2. 1-mistral-7B-GGUF | | 2023-11-27 | Roleplay okay, not censored | | | neural-chat-7B-v3-1-GGUF | | 2023-11-27 | Roleplay okay, not censored; some logic flaws | | | openchat_3. 5-16k-GGUF | | 2023-11-27 | Censored | | | Starling-LM-7B-alpha-GGUF | | 2023-11-29 | Censored; highly rated on Reddit | | | Orca-2-7B-GGUF | | 2023-11-29 | Censored | | | Orca

-2-13B-GGUF | | 2023-11-29 | Censored, some weird logic flaws worse than 7B version | | | MythoMist-7B-GGUF | | 2023-11-29 | Roleplay okay, not censored; some logic flaws | | | NeuralHermes-2. 5-Mistral-7B-GGUF | | 2023-12-05 | Roleplay good, not censored | | | stablelm-zephyr-3b-GGUF | | 2023-12-11 | 3B; roleplay ok; not censored; some logic flaws | | | deepseek-llm-7B-chat-GGUF | | 2023-12-11 | Censored | | | Mistral-7B-Instruct-v0. 2-GGUF | | 2023-12-13 | Partially censored; role play ok; highly

rated on Reddit | | | Mixtral-8x7B-Instruct-v0. 1-GGUF | | 2023-12-13 | MOE model; partially censored; role play ok | | | deepsex-34b-GGUF | | 2023-12-14 | 34B; NSFW model | | | phi-2 | | 2023-12-15 | 2. 7B; Base model; | | | Xwin-MLewd-13B-v0. 2-GGUF | | 2023-12-15 | 13B; NSFW model | | | MythoMax-L2-13B-GGUF | | 2023-12-15 | 13B; Censored; role play ok | | | LLaMA2-13B-Tiefighter-GGUF | | 2023-12-15 | 13B; role play good | | | LLaMA2-13B-Psyfighter2-GGUF | | 2023-12-15 | 13B

; Partially censored; role play ok; recommended on Reddit | | | Noromaid-13B-v0. 1-GGUF | | 2023-12-15 | 13B; NSFW model; roleplay good | | | dolphin-2. 5-mixtral-8x7b-GGUF | | 2023-12-20 | MOE model; claims to be uncensored but denies some requests; role play not usable | | | SOLAR-10. 7B; Censored; role play ok | | | Nous-Hermes-2-SOLAR-10. 7B; Partially Censored; role play good | | | openchat-3. 5-0106-GGUF | | 2024-01-12 | Role play and creative writing good; not censored | | | Mistral-7B

-Instruct-v0. 2-code-ft-GGUF | | 2024-01-12 | Coding model | | | Panda-7B-v0. 1-GGUF | | 2024-01-12 | Role

常见问题

记录了多少星标和分叉？

Troyanovsky/Local-LLM-Comparison-Colab-UI 有 1,107 个星标和 155 个分叉。

仓库排名是多少？

Troyanovsky/Local-LLM-Comparison-Colab-UI 排名第 11,571。

记录的语言和类别是什么？

Troyanovsky/Local-LLM-Comparison-Colab-UI 有 1,107 个星标，语言为 Jupyter Notebook，类别为 Lists。

哪个仓库排名最接近？

Troyanovsky/Local-LLM-Comparison-Colab-UI 与 Kim-Hammar/awesome-rl-for-cybersecurity 比较，后者排名第 11,586，有 1,103 个星标。

README

Local LLM Comparison & Colab Links (WIP)

(Update Nov. 27, 2023) The original goal of the repo was to compare some smaller models (7B and 13B) that can be run on consumer hardware so every model had a score for a set of questions from GPT-4. But I realized that as there are many more capable models appearing, the evaluation and comparison process may not suffice.

Therefore, I'm only putting Colab WebUI links for the newer models and you can try them out yourselves with a few clicks - after all, the effectiveness of a language model relies heavily on its suitability for your specific use case. By trying out the models firsthand, you can assess their performance and determine which one best fits your needs.

Newer Models without Scores:

These models can be run on consumer hardware and are generally good (from Reddit suggestions and my own experience). Try them out yourselves (tap the 'Open in Colab' button)!

Model Colab Link Date Added Notes Link
OpenHermes-2.5-Mistral-7B-GGUF 2023-11-27 Roleplay good, not censored https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF
zephyr-7B-beta-GGUF 2023-11-27 Roleplay okay, not censored https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF
dolphin-2.2.1-mistral-7B-GGUF 2023-11-27 Roleplay okay, not censored https://huggingface.co/TheBloke/dolphin-2.2.1-mistral-7B-GGUF
neural-chat-7B-v3-1-GGUF 2023-11-27 Roleplay okay, not censored; some logic flaws https://huggingface.co/TheBloke/neural-chat-7B-v3-1-GGUF
openchat_3.5-16k-GGUF 2023-11-27 Censored https://huggingface.co/TheBloke/openchat_3.5-16k-GGUF
Starling-LM-7B-alpha-GGUF 2023-11-29 Censored; highly rated on Reddit https://huggingface.co/TheBloke/Starling-LM-7B-alpha-GGUF
Orca-2-7B-GGUF 2023-11-29 Censored https://huggingface.co/TheBloke/Orca-2-7B-GGUF
Orca-2-13B-GGUF 2023-11-29 Censored, some weird logic flaws worse than 7B version https://huggingface.co/TheBloke/Orca-2-13B-GGUF
MythoMist-7B-GGUF 2023-11-29 Roleplay okay, not censored; some logic flaws https://huggingface.co/TheBloke/MythoMist-7B-GGUF
NeuralHermes-2.5-Mistral-7B-GGUF 2023-12-05 Roleplay good, not censored https://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-GGUF
stablelm-zephyr-3b-GGUF 2023-12-11 3B; roleplay ok; not censored; some logic flaws https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF
deepseek-llm-7B-chat-GGUF 2023-12-11 Censored https://huggingface.co/TheBloke/deepseek-llm-7B-chat-GGUF
Mistral-7B-Instruct-v0.2-GGUF 2023-12-13 Partially censored; role play ok; highly rated on Reddit https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF
Mixtral-8x7B-Instruct-v0.1-GGUF 2023-12-13 MOE model; partially censored; role play ok https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF
deepsex-34b-GGUF 2023-12-14 34B; NSFW model https://huggingface.co/TheBloke/deepsex-34b-GGUF
phi-2 2023-12-15 2.7B; Base model; https://huggingface.co/microsoft/phi-2
Xwin-MLewd-13B-v0.2-GGUF 2023-12-15 13B; NSFW model https://huggingface.co/TheBloke/Xwin-MLewd-13B-v0.2-GGUF
MythoMax-L2-13B-GGUF 2023-12-15 13B; Censored; role play ok https://huggingface.co/TheBloke/MythoMax-L2-13B-GGUF
LLaMA2-13B-Tiefighter-GGUF 2023-12-15 13B; role play good https://huggingface.co/TheBloke/LLaMA2-13B-Tiefighter-GGUF
LLaMA2-13B-Psyfighter2-GGUF 2023-12-15 13B; Partially censored; role play ok; recommended on Reddit https://huggingface.co/TheBloke/LLaMA2-13B-Psyfighter2-GGUF
Noromaid-13B-v0.1.1-GGUF 2023-12-15 13B; NSFW model; roleplay good https://huggingface.co/TheBloke/Noromaid-13B-v0.1.1-GGUF
dolphin-2.5-mixtral-8x7b-GGUF 2023-12-20 MOE model; claims to be uncensored but denies some requests; role play not usable https://huggingface.co/TheBloke/dolphin-2.5-mixtral-8x7b-GGUF
SOLAR-10.7B-Instruct-v1.0-GGUF 2023-12-21 10.7B; Censored; role play ok https://huggingface.co/TheBloke/SOLAR-10.7B-Instruct-v1.0-GGUF
Nous-Hermes-2-SOLAR-10.7B-GGUF 2024-01-08 10.7B; Partially Censored; role play good https://huggingface.co/TheBloke/Nous-Hermes-2-SOLAR-10.7B-GGUF
openchat-3.5-0106-GGUF 2024-01-12 Role play and creative writing good; not censored https://huggingface.co/TheBloke/openchat-3.5-0106-GGUF
Mistral-7B-Instruct-v0.2-code-ft-GGUF 2024-01-12 Coding model https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-code-ft-GGUF
Panda-7B-v0.1-GGUF 2024-01-12 Role play ok; creative writing good; partially censored https://huggingface.co/TheBloke/Panda-7B-v0.1-GGUF
LLaMA-Pro-8B-GGUF 2024-01-16 Claims to be good at math/coding; quality so so https://huggingface.co/TheBloke/LLaMA-Pro-8B-GGUF
dolphin-2.6-mistral-7B-dpo-laser-GGUF 2024-01-16 Uncensored; Good quality; role play good https://huggingface.co/TheBloke/dolphin-2.6-mistral-7B-dpo-laser-GGUF
laserxtral-GGUF 2024-01-17 4x7B MOE model; half memory than Mixtral; not censored; role play good https://huggingface.co/dagbs/laserxtral-GGUF
Mixtral_7Bx2_MoE-GGUF 2024-01-23 2x7B MOE model; uncensored; role play ok https://huggingface.co/TheBloke/Mixtral_7Bx2_MoE-GGUF
CapybaraHermes-2.5-Mistral-7B-GGUF 2024-02-06 Censored; https://huggingface.co/TheBloke/CapybaraHermes-2.5-Mistral-7B-GGUF
stable-code-3b-GGUF 2024-02-06 3B; Coding model; quality meh probably too small https://huggingface.co/TheBloke/stable-code-3b-GGUF/
EstopianMaid-13B-GGUF 2024-02-06 13B; Roleplay model https://huggingface.co/TheBloke/EstopianMaid-13B-GGUF
gemma-7b-it-GGUF 2024-02-28 7B; Censored https://huggingface.co/mlabonne/gemma-7b-it-GGUF/
StarCoder2-15B-GGUF 2024-03-20 15B; coding model; highly voted on r/LocalLLaMA https://huggingface.co/second-state/StarCoder2-15B-GGUF
Qwen1.5-7B-Chat-GGUF 2024-03-20 7B; Censored; highly voted on r/LocalLLaMA https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GGUF
Qwen1.5-14B-Chat-GGUF 2024-03-20 14B; Censored; highly voted on r/LocalLLaMA https://huggingface.co/Qwen/Qwen1.5-14B-Chat-GGUF
Hermes-2-Pro-Mistral-7B-GGUF 2024-03-22 7B; Not censored; role play ok; highly ranked on Huggingface https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF
Nous-Hermes-2-Mistral-7B-DPO-GGUF 2024-03-22 7B; Partially censored; role play good; highly ranked on Huggingface https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO-GGUF
stable-code-instruct-3b-GGUF 2024-03-27 3B; instruction tuned code generation model https://huggingface.co/bartowski/stable-code-instruct-3b-GGUF
Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4 2024-04-03 MOE; Small footprint; some logical errors https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4
Octopus-v2 2024-04-07 2B; non-quantized; optimized for on-device Android APIs https://huggingface.co/NexaAIDev/Octopus-v2/
codegemma-7b-it-GGUF 2024-04-18 7B; coding model https://huggingface.co/lmstudio-community/codegemma-7b-it-GGUF
CodeQwen1.5-7B-Chat-GGUF 2024-04-18 7B; coding model https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat-GGUF
WizardLM-2-7B-GGUF 2024-04-18 7B; Censored https://huggingface.co/MaziyarPanahi/WizardLM-2-7B-GGUF
Meta-Llama-3-8B-Instruct-GGUF 2024-04-19 8B; Censored https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF
dolphin-2.9-llama3-8b-gguff 2024-04-22 8B; Uncensored; logic seems degraded from Llama-3-8B https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b-gguff
Lexi-Llama-3-8B-Uncensored-GGUF 2024-04-24 8B; Uncensored https://huggingface.co/Orenguteng/Lexi-Llama-3-8B-Uncensored-GGUF
Llama3-8B-Chinese-Chat-GGUF 2024-04-24 8B; Chinese https://huggingface.co/QuantFactory/Llama3-8B-Chinese-Chat-GGUF
Phi-3-mini-4k-instruct-gguf 2024-04-24 3.8B; Censored; Fast https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf
Llama-3-8B-Instruct-32k-v0.1-GGUF 2024-04-25 8B; 32K context; good for summarization of long text https://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-32k-v0.1-GGUF
starcoder2-15b-instruct-v0.1-GGUF 2024-05-06 15B; coding model https://huggingface.co/bartowski/starcoder2-15b-instruct-v0.1-GGUF
Hermes-2-Pro-Llama-3-8B-GGUF 2024-05-06 8B; Partially censored; JSON, Tool Use, etc. https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF
Llama-3-ChatQA-1.5-8B-GGUF 2024-05-15 8B; Uncensored https://huggingface.co/bartowski/Llama-3-ChatQA-1.5-8B-GGUFF
Hermes-2-Theta-Llama-3-8B-GGUF 2024-05-17 8B; Censored; JSON, Tool Use, etc https://huggingface.co/NousResearch/Hermes-2-Theta-Llama-3-8B-GGUF
blossom-v5.1-9b-GGUF 2024-05-17 9B; Wizard/Orca/Math/Chinese/English mixed dataset https://huggingface.co/bartowski/blossom-v5.1-9b-GGUF
falcon-11B-GGUF 2024-05-17 11B; base raw pretrained model https://huggingface.co/bartowski/falcon-11B-GGUF
llama-3-cat-8b-instruct-v1-GGUFF 2024-05-20 8B; partially censored; system instruction fidelity; Role play https://huggingface.co/bartowski/llama-3-cat-8b-instruct-v1-GGUFF
Yi-1.5-9B-Chat-GGUF 2024-05-20 9B; Censored https://huggingface.co/bartowski/Yi-1.5-9B-Chat-GGUF
SFR-Iterative-DPO-LLaMA-3-8B-R-GGUF 2024-05-22 8B; Partially censored https://huggingface.co/bartowski/SFR-Iterative-DPO-LLaMA-3-8B-R-GGUF
Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-Imatrix 2024-05-22 8B; Roleplay; Trained to not decline requests; Seems to be trained on Quora data https://huggingface.co/Lewdiculous/Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-Imatrix
Mistral-7B-Instruct-v0.3-GGUF 2024-05-23 7B; Censored https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF
L3-8B-Stheno-v3.1-GGUF-IQ-Imatrix 2024-05-30 8B; trained for 1 on 1 uncensored roleplay https://huggingface.co/Lewdiculous/L3-8B-Stheno-v3.1-GGUF-IQ-Imatrix
aya-23-8B-GGUF 2024-05-30 8B; Censored https://huggingface.co/bartowski/aya-23-8B-GGUF
LLaMA3-iterative-DPO-final-GGUF 2024-05-30 8B; Censored https://huggingface.co/bartowski/LLaMA3-iterative-DPO-final-GGUF
openchat-3.6-8b-20240522-GGUF 2024-06-04 8B; Partially censored https://huggingface.co/bartowski/openchat-3.6-8b-20240522-GGUF
Meta-Llama-3-8B-Instruct-abliterated-v3-GGUF 2024-06-04 8B; Uncensored; https://huggingface.co/failspy/Meta-Llama-3-8B-Instruct-abliterated-v3-GGUF
NeuralDaredevil-8B-abliterated-GGUF 2024-06-19 8B; Uncensored; https://huggingface.co/QuantFactory/NeuralDaredevil-8B-abliterated-GGUF
Qwen2-7B-Instruct-GGUF 2024-06-24 7B; Censored https://huggingface.co/Qwen/Qwen2-7B-Instruct-GGUF
DeepSeek-Coder-V2-Lite-Instruct-GGUF 2024-06-27 16B; Instruct coding model https://huggingface.co/lmstudio-community/DeepSeek-Coder-V2-Lite-Instruct-GGUF
internlm2_5-7b-chat-gguf 2024-07-11 7B; Censored; Long context; Reasoning and Tool Use; https://huggingface.co/internlm/internlm2_5-7b-chat-gguf
gemma-2-9b-it-GGUF 2024-07-11 9B; Censored https://huggingface.co/bartowski/gemma-2-9b-it-GGUF
Smegmma-Deluxe-9B-v1-GGUF 2024-07-12 9B; Less censored gemma (still refuses some requests); Role play https://huggingface.co/bartowski/Smegmma-Deluxe-9B-v1-GGUF
h2o-danube3-4b-chat-GGUF 2024-07-17 4B; Partially censored https://huggingface.co/h2oai/h2o-danube3-4b-chat-GGUF
Tiger-Gemma-9B-v1-GGUF 2024-07-17 9B; Uncensored; but appears to have some flaws in logic https://huggingface.co/bartowski/Tiger-Gemma-9B-v1-GGUF
Gemmasutra-9B-v1-GGUF 2024-07-24 9B; Roleplay with some censorship https://huggingface.co/TheDrummer/Gemmasutra-9B-v1-GGUF
Meta-Llama-3.1-8B-Instruct-GGUF 2024-07-25 8B; Censored https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF
Mistral-Nemo-Instruct-2407-GGUF 2024-07-25 12.2B; Partially Uncensored; Recommended by Redditors https://huggingface.co/second-state/Mistral-Nemo-Instruct-2407-GGUF
Celeste-12B-V1.6_iMatrix_GGUF 2024-07-30 12B; Roleplay & story writing model, Uncensored https://huggingface.co/MarsupialAI/Celeste-12B-V1.6_iMatrix_GGUF
Hermes-3-Llama-3.1-8B-GGUF 2024-08-19 8B; Uncensored https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF
Gemma-2-9B-It-SPPO-Iter3-GGUF 2024-08-19 9B; Recommended on Reddit for summarization https://huggingface.co/bartowski/Gemma-2-9B-It-SPPO-Iter3-GGUF
Llama-3.1-Storm-8B-GGUF 2024-08-20 8B; Censored https://huggingface.co/bartowski/Llama-3.1-Storm-8B-GGUF
Phi-3.5-mini-instruct-GGUF 2024-08-21 3.5B; Censored, small & fast https://huggingface.co/lmstudio-community/Phi-3.5-mini-instruct-GGUF
Phi-3.5-mini-instruct_Uncensored-GGUF 2024-08-26 3.5B; Uncensored, small & fast, but has logic flaws & inconsistent results https://huggingface.co/bartowski/Phi-3.5-mini-instruct_Uncensored-GGUF
NemoMix-Unleashed-12B-GGUF 2024-08-26 12B; Partially censored, Roleplaying & story telling https://huggingface.co/bartowski/NemoMix-Unleashed-12B-GGUF
Mistral-NeMo-Minitron-8B-Base-GGUF 2024-08-26 8B; Base model https://huggingface.co/QuantFactory/Mistral-NeMo-Minitron-8B-Base-GGUF
Yi-Coder-9B-Chat-GGUF 2024-09-05 9B; Instruction coding model https://huggingface.co/bartowski/Yi-Coder-9B-Chat-GGUF
Llama-3.2-3B-Instruct-Q8_0-GGUF 2024-10-01 3B; Small censored model https://huggingface.co/hugging-quants/Llama-3.2-3B-Instruct-Q8_0-GGUF
Qwen2.5-7B-Instruct-GGUF 2024-10-01 7B; Censored, Recommended on r/LocalLLaMA https://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF
Qwen2.5-Coder-7B-Instruct-GGUF 2024-10-01 7B; Instruction coding model, recommended on r/LocalLLaMA https://huggingface.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF
Llama-3.1-8B-Lexi-Uncensored-V2-GGUF 2024-10-15 8B; Partially censored https://huggingface.co/Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2-GGUF
SuperNova-Medius-GGUF 2024-10-15 14B; Censored, Recommended on r/LocalLLaMA https://huggingface.co/arcee-ai/SuperNova-Medius-GGUF
Llama-3.2-3B-Instruct-uncensored-GGUF 2024-10-15 3B; Small, uncensored https://huggingface.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-GGUF
Ministral-8B-Instruct-2410-HF-GGUF-TEST 2024-10-21 8B; Partially censored https://huggingface.co/bartowski/Ministral-8B-Instruct-2410-HF-GGUF-TEST
granite-3.0-8b-instruct-GGUF 2024-10-28 8B; Censored https://huggingface.co/bartowski/granite-3.0-8b-instruct-GGUF
aya-expanse-8b-GGUF 2024-10-28 8B; Censored https://huggingface.co/bartowski/aya-expanse-8b-GGUF
Qwen2.5-Coder-14B-Instruct-GGUF 2024-11-12 14B; Coding model, good at its size https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct-GGUF
Marco-o1-GGUF 2024-11-25 7B; CoT model to mimic o1 for reasoning https://huggingface.co/bartowski/Marco-o1-GGUF
Llama-Sentient-3.2-3B-Instruct-GGUF 2024-11-25 3B; Censored https://huggingface.co/prithivMLmods/Llama-Sentient-3.2-3B-Instruct-GGUF
Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-GGUF 2024-12-05 10B; Uncensored story writing model https://huggingface.co/DavidAU/Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-GGUF
INTELLECT-1-Instruct-GGUF 2024-12-05 10B; Censored https://huggingface.co/lmstudio-community/INTELLECT-1-Instruct-GGUF
L3.1-RP-Hero-BigTalker-8B-GGUF 2024-12-05 8B; Uncensored NSFW roleplay model https://huggingface.co/DavidAU/L3.1-RP-Hero-BigTalker-8B-GGUF
EXAONE-3.5-7.8B-Instruct-GGUF 2024-12-09 7.8B; Censored, English and Korean, context up to 32K https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-GGUF
Virtuoso-Small-GGUF 2024-12-10 14B; Censored, good for general tasks, recommended on Reddit https://huggingface.co/bartowski/Virtuoso-Small-GGUF
phi-4 2024-12-16 14B; Censored https://huggingface.co/matteogeniaccio/phi-4
Hermes-3-Llama-3.2-3B-GGUF 2024-12-21 3B; Partially censored https://huggingface.co/NousResearch/Hermes-3-Llama-3.2-3B-GGUF
Falcon3-Mamba-7B-Instruct-GGUF 2024-12-21 7B; Mamba1 architecture, 32K context https://huggingface.co/tiiuae/Falcon3-Mamba-7B-Instruct-GGUF
QwQ-LCoT-7B-Instruct-GGUF 2024-12-21 7B; finetuned on CoT https://huggingface.co/prithivMLmods/QwQ-LCoT-7B-Instruct-GGUF
granite-3.1-8b-instruct-GGUF 2024-12-24 8B; multilingual, 128K context https://huggingface.co/bartowski/granite-3.1-8b-instruct-GGUF
granite-3.1-3b-a800m-instruct-GGUF 2024-12-24 3B; MoE, 128K context https://huggingface.co/bartowski/granite-3.1-3b-a800m-instruct-GGUF
Falcon3-10B-Instruct-GGUF 2025-01-21 10B; 32K context, English/French/Spanish/Portuguese https://huggingface.co/tiiuae/Falcon3-10B-Instruct-GGUF
DeepSeek-R1-Distill-Llama-8B-GGUF 2025-01-21 8B; Reasoning model, 128K context https://huggingface.co/bartowski/DeepSeek-R1-Distill-Llama-8B-GGUF
DeepSeek-R1-Distill-Qwen-7B-GGUF 2025-02-06 7B; Reasoning model, 128K context, censored https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF
DeepSeek-R1-Distill-Qwen-14B-GGUF 2025-02-06 14B; Reasoning model, 128K context, censored https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF
Xwen-7B-Chat-GGUF 2025-02-06 7B; Uncensored https://huggingface.co/mradermacher/Xwen-7B-Chat-GGUF
Virtuoso-Small-v2-GGUF 2025-02-06 14B; Censored https://huggingface.co/bartowski/arcee-ai_Virtuoso-Small-v2-GGUF
DeepHermes-3-Llama-3-8B-Preview-GGUF 2025-02-14 8B; Reasoning model, partially censored https://huggingface.co/NousResearch/DeepHermes-3-Llama-3-8B-Preview-GGUF
Reverb-7b-Q4_K_M-GGUF 2025-02-20 7B; Censored https://huggingface.co/ozone-ai/Reverb-7b-Q4_K_M-GGUF
Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF 2025-03-03 8x3B; MoE, Uncensored NSFW, roleplay, writing https://huggingface.co/DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF
google_gemma-3-12b-it-GGUF 2025-03-17 12B; Censored https://huggingface.co/bartowski/google_gemma-3-12b-it-GGUF
microsoft_Phi-4-mini-instruct-GGUF 2025-03-17 3.8B; Censored, 128 K context https://huggingface.co/bartowski/microsoft_Phi-4-mini-instruct-GGUF
RekaAI_reka-flash-3-GGUF 2025-03-18 21B; Reasoning model, censored https://huggingface.co/bartowski/RekaAI_reka-flash-3-GGUF
mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF 2025-03-19 24B; Censored, unlike previous mistral models https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF
agentica-org_DeepCoder-14B-Preview-GGUF 2025-04-09 14B; Reasoning coding model https://huggingface.co/bartowski/agentica-org_DeepCoder-14B-Preview-GGUF
deepcogito_cogito-v1-preview-qwen-14B-GGUF 2025-04-09 14B; Censored https://huggingface.co/bartowski/deepcogito_cogito-v1-preview-qwen-14B-GGUF
Qwen_Qwen3-14B-GGUF 2025-04-29 14B; Censored https://huggingface.co/bartowski/Qwen_Qwen3-14B-GGUF
ReadyArt_Amoral-Fallen-Omega-Gemma3-12B-GGUF 2025-04-29 12B; Uncensored NSFW roleplay, but may deny non-sexual illegal requests https://huggingface.co/bartowski/ReadyArt_Amoral-Fallen-Omega-Gemma3-12B-GGUF
soob3123_Veritas-12B-GGUF 2025-04-29 12B; Partially censored https://huggingface.co/bartowski/soob3123_Veritas-12B-GGUF
THUDM_GLM-4-9B-0414-GGUF 2025-04-29 9B; Censored https://huggingface.co/bartowski/THUDM_GLM-4-9B-0414-GGUF
microsoft_Phi-4-reasoning-plus-GGUF 2025-05-06 14B; Reasoning model, Censored https://huggingface.co/bartowski/microsoft_Phi-4-reasoning-plus-GGUF
Goekdeniz-Guelmez_Josiefied-Qwen3-8B-abliterated-v1-GGUF 2025-05-06 8B; Uncensored model https://huggingface.co/bartowski/Goekdeniz-Guelmez_Josiefied-Qwen3-8B-abliterated-v1-GGUF
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF 2025-06-09 8B; Reasoning model, Censored https://huggingface.co/bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
open-thoughts_OpenThinker3-7B-GGUF 2025-06-09 7B; Reasoning model, Uncensored, tends to think a lot before answering https://huggingface.co/bartowski/open-thoughts_OpenThinker3-7B-GGUF
Menlo_Jan-nano-128k-GGUF 2025-06-26 4B; Long-context model optimized for MCP use and deep research https://huggingface.co/bartowski/Menlo_Jan-nano-128k-GGUF
google_gemma-3n-E4B-it-GGUF 2025-07-10 8B with 4B activated; Censored, 32K context, efficient parameter management https://huggingface.co/bartowski/google_gemma-3n-E4B-it-GGUF
janhq_Jan-v1-4B-GGUF 2025-08-15 4B; Optimized for search and deep research https://huggingface.co/bartowski/janhq_Jan-v1-4B-GGUF
Qwen3-Coder-30B-A3B-Instruct-GGUF 2025-08-15 30B with 3 B activated; Coding model https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF
Qwen3-30B-A3B-Instruct-2507-GGUF 2025-08-15 30B with 3 B activated; Censored https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF
Qwen3-4B-Instruct-2507-GGUF 2025-08-15 4B; Censored https://huggingface.co/unsloth/Qwen3-4B-Instruct-2507-GGUF
NousResearch_Hermes-4-14B-GGUF 2025-09-09 14B; Partially censored https://huggingface.co/bartowski/NousResearch_Hermes-4-14B-GGUF
VibeThinker-1.5B-GGUFF 2025-11-24 1.5B; Math and Algo Coding model https://huggingface.co/MaziyarPanahi/VibeThinker-1.5B-GGUFF
Ministral-3-14B-Instruct-2512-GGUF 2025-12-08 14B; Multi-modal, Agentic, Censored model https://huggingface.co/unsloth/Ministral-3-14B-Instruct-2512-GGUF
Ministral-3-14B-Reasoning-2512-GGUF 2025-12-08 14B; Reasoning, Multi-modal, Agentic, Censored model https://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-GGUF
NousResearch_NousCoder-14B-GGUF 2026-01-13 14B; Coding model https://huggingface.co/bartowski/NousResearch_NousCoder-14B-GGUF
nvidia_Nemotron-Cascade-14B-Thinking-GGUF 2026-01-13 15B; Coding, reasoning model https://huggingface.co/bartowski/nvidia_Nemotron-Cascade-14B-Thinking-GGUF

Star History

Share This Project

If you find this project helpful, please share it:

Quick Share Links:

You may also be interested in

Check out awesome-TTS-Colab for a curated list of Google Colab notebooks featuring open source text-to-speech (TTS) models. These notebooks are easy to use and can also be run on your local machine!

Older Models with Scores

Older Models with Scores:

These models work better among the models I tested on my hardware (i5-12490F, 32GB RAM, RTX 3060 Ti GDDR6X 8GB VRAM): (Note: Because llama.cpp has made some breaking changes to the support of older ggml models. Some older ggml versions listed below may not work properly on current llama.cpp. But there should be GPTQ equivalents or newer ggml versions for the models.)

NOTE:

  • Major edit on June 30, 2023. Since I noticed GPT-4 started to give more granular scores and tends to give higher scores, I have decided to re-run the tests for all models so they can be compared (All evaluations done with GPT-4-0613 for consistency). The scores in the table below are the average of the scores from the latest tests. The scores in the table below are not comparable to the scores in the previous versions of this README.
Model_Name Avg_Score Colab_Link Date_Added Link
Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) 10.00 2023/10/08 https://huggingface.co/TheBloke/Mistral-7B-OpenOrca-GGUF
Llama-2-13B-chat (using oobabooga/text-generation-webui) 9.65 2023/07/20 https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML
wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) 9.63 2023/05/07 https://huggingface.co/TheBloke/wizard-vicuna-13B-GGML
Nous-Capybara-7B (using oobabooga/text-generation-webui) 9.56 2023/10/08 https://huggingface.co/TheBloke/Nous-Capybara-7B-GGUF
vicuna-13B-v1.5 (using oobabooga/text-generation-webui) 9.53 2023/08/09 https://huggingface.co/TheBloke/vicuna-13B-v1.5-GGML
wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) 9.53 2023/05/29 https://huggingface.co/TheBloke/wizardLM-13B-1.0-GPTQ
airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) 9.50 2023/06/30 https://huggingface.co/TheBloke/airoboros-13B-gpt4-1.4-GPTQ
Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) 9.44 2023/06/03 https://huggingface.co/TheBloke/Nous-Hermes-13B-GPTQ/tree/main
Dolphin-Llama-13B (using oobabooga/text-generation-webui) 9.38 2023/07/24 https://huggingface.co/TheBloke/Dolphin-Llama-13B-GGML
Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) 9.37 2023/10/08 https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF
OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) 9.37 2023/08/15 https://huggingface.co/TheBloke/OpenOrca-Platypus2-13B-GGML
airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) 9.34 2023/08/01 https://huggingface.co/TheBloke/airoboros-l2-13b-gpt4-2.0-GGML
Chronos-13B-v2 (using oobabooga/text-generation-webui) 9.31 2023/08/09 https://huggingface.co/TheBloke/Chronos-13B-v2-GGML
vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) 9.31 2023/06/29 https://huggingface.co/TheBloke/vicuna-13b-v1.3.0-GPTQ
MythoLogic-13B (using oobabooga/text-generation-webui) 9.31 2023/07/20 https://huggingface.co/TheBloke/MythoLogic-13B-GGML
Selfee-13B-GPTQ (using oobabooga/text-generation-webui) 9.28 2023/06/07 https://huggingface.co/TheBloke/Selfee-13B-GPTQ
WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) 9.28 2023/07/26 https://huggingface.co/TheBloke/WizardLM-13B-V1.2-GGML
minotaur-13B-GPTQ (using oobabooga/text-generation-webui) 9.28 2023/06/09 https://huggingface.co/TheBloke/minotaur-13B-GPTQ
Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) 9.20 2023/09/21 https://huggingface.co/TheBloke/Pygmalion-2-13B-SuperCOT2-GGUF
Athena-v1 (using oobabooga/text-generation-webui) 9.19 2023/08/31 https://huggingface.co/TheBloke/Athena-v1-GGUF
PuddleJumper-13B (using oobabooga/text-generation-webui) 9.1875 2023/08/29 https://huggingface.co/TheBloke/PuddleJumper-13B-GGUF
Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) 9.17 2023/07/24 https://huggingface.co/TheBloke/Nous-Hermes-Llama2-GGML
Luban-13B (using oobabooga/text-generation-webui) 9.15 2023/08/31 https://huggingface.co/TheBloke/Luban-13B-GGUF
MythoBoros-13B (using oobabooga/text-generation-webui) 9.15 2023/07/24 https://huggingface.co/TheBloke/MythoBoros-13B-GGML
13B-Ouroboros (using oobabooga/text-generation-webui) 9.11 2023/07/27 https://huggingface.co/TheBloke/13B-Ouroboros-GGML
tulu-13B-GPTQ (using oobabooga/text-generation-webui) 9.06 2023/06/13 https://huggingface.co/TheBloke/tulu-13B-GPTQ
AlpacaCielo-13B (using oobabooga/text-generation-webui) 9.03125 2023/07/27 https://huggingface.co/TheBloke/AlpacaCielo-13B-GGML
StableBeluga-13B (using oobabooga/text-generation-webui) 9 2023/08/01 https://huggingface.co/TheBloke/StableBeluga-13B-GGML
Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) 8.97 2023/08/10 https://huggingface.co/TheBloke/Chronos-Hermes-13B-v2-GGML
OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) 8.97 2023/09/05 https://huggingface.co/TheBloke/OpenBuddy-Llama2-13B-v11.1-GGUF
Camel-Platypus2-13B (using oobabooga/text-generation-webui) 8.94 2023/08/15 https://huggingface.co/TheBloke/Camel-Platypus2-13B-GGML
airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) 8.94 2023/09/21 https://huggingface.co/TheBloke/airoboros-l2-13b-gpt4-m2.0-GGUF
UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) 8.89 2023/06/30 https://huggingface.co/TheBloke/UltraLM-13B-GPTQ
13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) 8.88 2023/06/03 https://huggingface.co/digitous/13B-HyperMantis_GPTQ_4bit-128g/
Stable-Platypus2-13B (using oobabooga/text-generation-webui) 8.875 2023/08/15 https://huggingface.co/TheBloke/Stable-Platypus2-13B-GGML
Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) 8.84 2023/05/25 https://huggingface.co/TheBloke/airoboros-13B-GPTQ
Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) 8.84 2023/09/21 https://huggingface.co/TheBloke/Kuchiki-1.1-L2-7B-GGUF
WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) 8.80625 2023/08/09 https://huggingface.co/TheBloke/WizardLM-1.0-Uncensored-Llama2-13B-GGML
Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) 8.75 2023/08/10 https://huggingface.co/TheBloke/Chronos-Beluga-v2-13B-GGML
Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) 8.75 2023/06/09 https://huggingface.co/TheBloke/Vicuna-13B-CoT-GPTQ
wizardLM-7B.q4_2 (in GPT4All) 8.75 No 2023/05/07 https://gpt4all.io/models/ggml-wizardLM-7B.q4_2.bin
OpenChat_v3.2 (using oobabooga/text-generation-webui) 8.71875 2023/08/01 https://huggingface.co/TheBloke/OpenChat_v3.2-GGML
Huginn-13B (using oobabooga/text-generation-webui) 8.7125 2023/08/10 https://huggingface.co/TheBloke/Huginn-13B-GGML
WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) 8.66 2023/07/17 https://huggingface.co/TheBloke/WizardLM-13B-V1.1-GGML
robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) 8.66 2023/06/19 https://huggingface.co/TheBloke/robin-13B-v2-GPTQ
llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) 8.625 2023/07/21 https://huggingface.co/TheBloke/llama-2-13B-Guanaco-QLoRA-GGML
mpt-7b-chat (in GPT4All) 8.53 No 2023/05/11 https://gpt4all.io/models/ggml-mpt-7b-chat.bin
chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) 8.48125 2023/06/16 https://huggingface.co/TheBloke/chronos-hermes-13B-GPTQ
Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) 8.46875 2023/07/20 https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGML
stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) 8.25 2023/05/12 https://huggingface.co/TheBloke/stable-vicuna-13B-GPTQ
manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) 8.21875 2023/05/24 https://huggingface.co/TheBloke/manticore-13b-chat-pyg-GPTQ
CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) 8.09375 2023/06/10 https://huggingface.co/TheBloke/CAMEL-13B-Combined-Data-GPTQ
WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) 8.09375 2023/06/02 https://huggingface.co/TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ
llama-13b-supercot-GGML (using oobabooga/text-generation-webui) 8.01 2023/07/05 https://huggingface.co/TheBloke/llama-13b-supercot-GGML
Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) 7.96875 2023/05/24 https://huggingface.co/TheBloke/Project-Baize-v2-13B-GPTQ
koala-13B-4bit-128g.GGML (using llama.cpp) 7.9375 No 2023/05/07 https://huggingface.co/TheBloke/koala-13B-GPTQ-4bit-128g-GGML
wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) 7.90625 2023/05/19 https://huggingface.co/4bit/WizardLM-13B-Uncensored-4bit-128g
vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) 7.875 2023/06/29 https://huggingface.co/TheBloke/vicuna-7B-v1.3-GPTQ
Manticore-13B-GPTQ (using oobabooga/text-generation-webui) 7.78125 2023/05/23 https://huggingface.co/TheBloke/Manticore-13B-GPTQ
vicuna-13b-1.1-q4_2 (in GPT4All) 7.75 No 2023/05/07 https://gpt4all.io/models/ggml-vicuna-13b-1.1-q4_2.bin
falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) 7.625 2023/06/02 https://huggingface.co/TheBloke/falcon-7b-instruct-GPTQ
guanaco-13B-GPTQ (using oobabooga/text-generation-webui) 7.5625 2023/05/26 https://huggingface.co/TheBloke/guanaco-13B-GPTQ
Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) 7.31 2023/08/31 https://huggingface.co/TheBloke/Mythical-Destroyer-V2-L2-13B-GGUF
Kimiko-v2-13B (using oobabooga/text-generation-webui) 7.25 2023/08/31 https://huggingface.co/TheBloke/Kimiko-v2-13B-GGUF
orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) 7.0875 2023/06/28 https://huggingface.co/TheBloke/orca_mini_13B-GGML
Platypus2-13B (using oobabooga/text-generation-webui) 7.03125 2023/08/15 https://huggingface.co/TheBloke/Platypus2-13B-GGML
Redmond-Puffin-13B (using oobabooga/text-generation-webui) 7.03125 <
排名相邻
同类项目