AI Rank
中文
← Open Source
Troyanovsky

Local-LLM-Comparison-Colab-UI

Compare the performance of different LLM that can be deployed locally on consumer hardware. Run yourself with Colab WebUI.

ListsModel collectionsJupyter Notebook
Open on GitHub
Momentum
+0stars in 24 hours0.0%
1.11k
Stars
155
Forks
+2
This week
3
Contributors
Created 2023-05-07 · Updated 2026-10-14 · #11571 today
Top developers
TroyanovskyklipskiRonbalt

Troyanovsky/Local-LLM-Comparison-Colab-UI is ranked 11,571 with 1,107 stars.

Same-category comparison

Troyanovsky/Local-LLM-Comparison-Colab-UI is ranked 11,571 among these 8 peers; Kim-Hammar/awesome-rl-for-cybersecurity is ranked 11,586.

Same-category comparison
NameRankMetrics
leomaurodesenv/game-datasets11,465
Stars:
1,134
Forks:
94
Description:
:video_game: A curated list of awesome game datasets, and tools to artificial intelligence in games.
offchan42/machine-learning-curriculum11,468
Stars:
1,133
Forks:
253
Description:
:computer: Learn to make machines learn so that you don't have to struggle to program them; The ultimate list.
Shujian2015/FreeML11,470
Stars:
1,132
Forks:
511
Description:
A List of Data Science/Machine Learning Resources (Mostly Free).
TSCenter/awesome-time-series-papers11,477
Stars:
1,130
Forks:
54
Description:
An Awesome List of the latest time series papers and code from top AI venues.
Kim-Hammar/awesome-rl-for-cybersecurity11,586
Stars:
1,103
Forks:
146
Description:
A curated list of resources dedicated to reinforcement learning applied to cyber security.
Chen-Cai-OSU/awesome-equivariant-network11,601
Stars:
1,100
Forks:
108
Description:
Paper list for equivariant neural network.
qualisero/awesome-pi-agent11,606
Language:
JavaScript
Stars:
1,100
Forks:
66
Description:
Awesome list of add-ons, hooks, tools, skills, and resources for the pi coding agent (pi-mono).
abubakarsiddik31/claude-skills-collection11,608
Stars:
1,099
Forks:
204
Description:
A curated collection of official and community-built Claude Skills – extend Anthropic's Claude with powerful, modular capabilities for productivity, creativity, coding, and more.
Key parameters and specifications
Key parameters and specifications
ParameterValue
LanguageJupyter Notebook
CategoryLists
SubcategoryModel collections
KeywordsLocalAI
Stars1,107
Forks155
Contributors3
Created date2023-05-07
Updated date2026-10-14
Stars in 1 day0
Stars in 7 days2
Rank11,571
Previous rank5,997
Rank change-5,574
7-day stars rank8,228
Previous 7-day stars rank8,297
7-day stars rank change69
Total stars rank10,304
Previous total stars rank10,293
Total stars rank change-11
README summary

Local LLM Comparison & Colab Links (WIP) (Update Nov. 27, 2023) The original goal of the repo was to compare some smaller models (7B and 13B) that can be run on consumer hardware so every model had a score for a set of questions from GPT-4. But I realized that as there are many more capable models appearing, the evaluation and comparison process may not suffice. Therefore, I'm only putting Colab WebUI links for the newer models and you can

try them out yourselves with a few clicks - after all, the effectiveness of a language model relies heavily on its suitability for your specific use case. By trying out the models firsthand, you can assess their performance and determine which one best fits your needs. Newer Models without Scores: These models can be run on consumer hardware and are generally good (from Reddit suggestions and my own experience). Try them out yourselves (tap the 'Open in Colab' button)! 5-Mistral

-7B-GGUF | | 2023-11-27 | Roleplay good, not censored | | | zephyr-7B-beta-GGUF | | 2023-11-27 | Roleplay okay, not censored | | | dolphin-2. 1-mistral-7B-GGUF | | 2023-11-27 | Roleplay okay, not censored | | | neural-chat-7B-v3-1-GGUF | | 2023-11-27 | Roleplay okay, not censored; some logic flaws | | | openchat_3. 5-16k-GGUF | | 2023-11-27 | Censored | | | Starling-LM-7B-alpha-GGUF | | 2023-11-29 | Censored; highly rated on Reddit | | | Orca-2-7B-GGUF | | 2023-11-29 | Censored | | | Orca

-2-13B-GGUF | | 2023-11-29 | Censored, some weird logic flaws worse than 7B version | | | MythoMist-7B-GGUF | | 2023-11-29 | Roleplay okay, not censored; some logic flaws | | | NeuralHermes-2. 5-Mistral-7B-GGUF | | 2023-12-05 | Roleplay good, not censored | | | stablelm-zephyr-3b-GGUF | | 2023-12-11 | 3B; roleplay ok; not censored; some logic flaws | | | deepseek-llm-7B-chat-GGUF | | 2023-12-11 | Censored | | | Mistral-7B-Instruct-v0. 2-GGUF | | 2023-12-13 | Partially censored; role play ok; highly

rated on Reddit | | | Mixtral-8x7B-Instruct-v0. 1-GGUF | | 2023-12-13 | MOE model; partially censored; role play ok | | | deepsex-34b-GGUF | | 2023-12-14 | 34B; NSFW model | | | phi-2 | | 2023-12-15 | 2. 7B; Base model; | | | Xwin-MLewd-13B-v0. 2-GGUF | | 2023-12-15 | 13B; NSFW model | | | MythoMax-L2-13B-GGUF | | 2023-12-15 | 13B; Censored; role play ok | | | LLaMA2-13B-Tiefighter-GGUF | | 2023-12-15 | 13B; role play good | | | LLaMA2-13B-Psyfighter2-GGUF | | 2023-12-15 | 13B

; Partially censored; role play ok; recommended on Reddit | | | Noromaid-13B-v0. 1-GGUF | | 2023-12-15 | 13B; NSFW model; roleplay good | | | dolphin-2. 5-mixtral-8x7b-GGUF | | 2023-12-20 | MOE model; claims to be uncensored but denies some requests; role play not usable | | | SOLAR-10. 7B; Censored; role play ok | | | Nous-Hermes-2-SOLAR-10. 7B; Partially Censored; role play good | | | openchat-3. 5-0106-GGUF | | 2024-01-12 | Role play and creative writing good; not censored | | | Mistral-7B

-Instruct-v0. 2-code-ft-GGUF | | 2024-01-12 | Coding model | | | Panda-7B-v0. 1-GGUF | | 2024-01-12 | Role

Frequently asked questions

How many stars and forks are recorded?

Troyanovsky/Local-LLM-Comparison-Colab-UI has 1,107 stars and 155 forks.

What is the repository rank?

Troyanovsky/Local-LLM-Comparison-Colab-UI is ranked 11,571.

What language and category are recorded?

Troyanovsky/Local-LLM-Comparison-Colab-UI has 1,107 stars, uses Jupyter Notebook and belongs to Lists.

Which repository is closest in rank?

Troyanovsky/Local-LLM-Comparison-Colab-UI is compared with Kim-Hammar/awesome-rl-for-cybersecurity, ranked 11,586 with 1,103 stars.

README

Local LLM Comparison & Colab Links (WIP)

(Update Nov. 27, 2023) The original goal of the repo was to compare some smaller models (7B and 13B) that can be run on consumer hardware so every model had a score for a set of questions from GPT-4. But I realized that as there are many more capable models appearing, the evaluation and comparison process may not suffice.

Therefore, I'm only putting Colab WebUI links for the newer models and you can try them out yourselves with a few clicks - after all, the effectiveness of a language model relies heavily on its suitability for your specific use case. By trying out the models firsthand, you can assess their performance and determine which one best fits your needs.

Newer Models without Scores:

These models can be run on consumer hardware and are generally good (from Reddit suggestions and my own experience). Try them out yourselves (tap the 'Open in Colab' button)!

Model Colab Link Date Added Notes Link
OpenHermes-2.5-Mistral-7B-GGUF 2023-11-27 Roleplay good, not censored https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF
zephyr-7B-beta-GGUF 2023-11-27 Roleplay okay, not censored https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF
dolphin-2.2.1-mistral-7B-GGUF 2023-11-27 Roleplay okay, not censored https://huggingface.co/TheBloke/dolphin-2.2.1-mistral-7B-GGUF
neural-chat-7B-v3-1-GGUF 2023-11-27 Roleplay okay, not censored; some logic flaws https://huggingface.co/TheBloke/neural-chat-7B-v3-1-GGUF
openchat_3.5-16k-GGUF 2023-11-27 Censored https://huggingface.co/TheBloke/openchat_3.5-16k-GGUF
Starling-LM-7B-alpha-GGUF 2023-11-29 Censored; highly rated on Reddit https://huggingface.co/TheBloke/Starling-LM-7B-alpha-GGUF
Orca-2-7B-GGUF 2023-11-29 Censored https://huggingface.co/TheBloke/Orca-2-7B-GGUF
Orca-2-13B-GGUF 2023-11-29 Censored, some weird logic flaws worse than 7B version https://huggingface.co/TheBloke/Orca-2-13B-GGUF
MythoMist-7B-GGUF 2023-11-29 Roleplay okay, not censored; some logic flaws https://huggingface.co/TheBloke/MythoMist-7B-GGUF
NeuralHermes-2.5-Mistral-7B-GGUF 2023-12-05 Roleplay good, not censored https://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-GGUF
stablelm-zephyr-3b-GGUF 2023-12-11 3B; roleplay ok; not censored; some logic flaws https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF
deepseek-llm-7B-chat-GGUF 2023-12-11 Censored https://huggingface.co/TheBloke/deepseek-llm-7B-chat-GGUF
Mistral-7B-Instruct-v0.2-GGUF 2023-12-13 Partially censored; role play ok; highly rated on Reddit https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF
Mixtral-8x7B-Instruct-v0.1-GGUF 2023-12-13 MOE model; partially censored; role play ok https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF
deepsex-34b-GGUF 2023-12-14 34B; NSFW model https://huggingface.co/TheBloke/deepsex-34b-GGUF
phi-2 2023-12-15 2.7B; Base model; https://huggingface.co/microsoft/phi-2
Xwin-MLewd-13B-v0.2-GGUF 2023-12-15 13B; NSFW model https://huggingface.co/TheBloke/Xwin-MLewd-13B-v0.2-GGUF
MythoMax-L2-13B-GGUF 2023-12-15 13B; Censored; role play ok https://huggingface.co/TheBloke/MythoMax-L2-13B-GGUF
LLaMA2-13B-Tiefighter-GGUF 2023-12-15 13B; role play good https://huggingface.co/TheBloke/LLaMA2-13B-Tiefighter-GGUF
LLaMA2-13B-Psyfighter2-GGUF 2023-12-15 13B; Partially censored; role play ok; recommended on Reddit https://huggingface.co/TheBloke/LLaMA2-13B-Psyfighter2-GGUF
Noromaid-13B-v0.1.1-GGUF 2023-12-15 13B; NSFW model; roleplay good https://huggingface.co/TheBloke/Noromaid-13B-v0.1.1-GGUF
dolphin-2.5-mixtral-8x7b-GGUF 2023-12-20 MOE model; claims to be uncensored but denies some requests; role play not usable https://huggingface.co/TheBloke/dolphin-2.5-mixtral-8x7b-GGUF
SOLAR-10.7B-Instruct-v1.0-GGUF 2023-12-21 10.7B; Censored; role play ok https://huggingface.co/TheBloke/SOLAR-10.7B-Instruct-v1.0-GGUF
Nous-Hermes-2-SOLAR-10.7B-GGUF 2024-01-08 10.7B; Partially Censored; role play good https://huggingface.co/TheBloke/Nous-Hermes-2-SOLAR-10.7B-GGUF
openchat-3.5-0106-GGUF 2024-01-12 Role play and creative writing good; not censored https://huggingface.co/TheBloke/openchat-3.5-0106-GGUF
Mistral-7B-Instruct-v0.2-code-ft-GGUF 2024-01-12 Coding model https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-code-ft-GGUF
Panda-7B-v0.1-GGUF 2024-01-12 Role play ok; creative writing good; partially censored https://huggingface.co/TheBloke/Panda-7B-v0.1-GGUF
LLaMA-Pro-8B-GGUF 2024-01-16 Claims to be good at math/coding; quality so so https://huggingface.co/TheBloke/LLaMA-Pro-8B-GGUF
dolphin-2.6-mistral-7B-dpo-laser-GGUF 2024-01-16 Uncensored; Good quality; role play good https://huggingface.co/TheBloke/dolphin-2.6-mistral-7B-dpo-laser-GGUF
laserxtral-GGUF 2024-01-17 4x7B MOE model; half memory than Mixtral; not censored; role play good https://huggingface.co/dagbs/laserxtral-GGUF
Mixtral_7Bx2_MoE-GGUF 2024-01-23 2x7B MOE model; uncensored; role play ok https://huggingface.co/TheBloke/Mixtral_7Bx2_MoE-GGUF
CapybaraHermes-2.5-Mistral-7B-GGUF 2024-02-06 Censored; https://huggingface.co/TheBloke/CapybaraHermes-2.5-Mistral-7B-GGUF
stable-code-3b-GGUF 2024-02-06 3B; Coding model; quality meh probably too small https://huggingface.co/TheBloke/stable-code-3b-GGUF/
EstopianMaid-13B-GGUF 2024-02-06 13B; Roleplay model https://huggingface.co/TheBloke/EstopianMaid-13B-GGUF
gemma-7b-it-GGUF 2024-02-28 7B; Censored https://huggingface.co/mlabonne/gemma-7b-it-GGUF/
StarCoder2-15B-GGUF 2024-03-20 15B; coding model; highly voted on r/LocalLLaMA https://huggingface.co/second-state/StarCoder2-15B-GGUF
Qwen1.5-7B-Chat-GGUF 2024-03-20 7B; Censored; highly voted on r/LocalLLaMA https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GGUF
Qwen1.5-14B-Chat-GGUF 2024-03-20 14B; Censored; highly voted on r/LocalLLaMA https://huggingface.co/Qwen/Qwen1.5-14B-Chat-GGUF
Hermes-2-Pro-Mistral-7B-GGUF 2024-03-22 7B; Not censored; role play ok; highly ranked on Huggingface https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF
Nous-Hermes-2-Mistral-7B-DPO-GGUF 2024-03-22 7B; Partially censored; role play good; highly ranked on Huggingface https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO-GGUF
stable-code-instruct-3b-GGUF 2024-03-27 3B; instruction tuned code generation model https://huggingface.co/bartowski/stable-code-instruct-3b-GGUF
Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4 2024-04-03 MOE; Small footprint; some logical errors https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4
Octopus-v2 2024-04-07 2B; non-quantized; optimized for on-device Android APIs https://huggingface.co/NexaAIDev/Octopus-v2/
codegemma-7b-it-GGUF 2024-04-18 7B; coding model https://huggingface.co/lmstudio-community/codegemma-7b-it-GGUF
CodeQwen1.5-7B-Chat-GGUF 2024-04-18 7B; coding model https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat-GGUF
WizardLM-2-7B-GGUF 2024-04-18 7B; Censored https://huggingface.co/MaziyarPanahi/WizardLM-2-7B-GGUF
Meta-Llama-3-8B-Instruct-GGUF 2024-04-19 8B; Censored https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF
dolphin-2.9-llama3-8b-gguff 2024-04-22 8B; Uncensored; logic seems degraded from Llama-3-8B https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b-gguff
Lexi-Llama-3-8B-Uncensored-GGUF 2024-04-24 8B; Uncensored https://huggingface.co/Orenguteng/Lexi-Llama-3-8B-Uncensored-GGUF
Llama3-8B-Chinese-Chat-GGUF 2024-04-24 8B; Chinese https://huggingface.co/QuantFactory/Llama3-8B-Chinese-Chat-GGUF
Phi-3-mini-4k-instruct-gguf 2024-04-24 3.8B; Censored; Fast https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf
Llama-3-8B-Instruct-32k-v0.1-GGUF 2024-04-25 8B; 32K context; good for summarization of long text https://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-32k-v0.1-GGUF
starcoder2-15b-instruct-v0.1-GGUF 2024-05-06 15B; coding model https://huggingface.co/bartowski/starcoder2-15b-instruct-v0.1-GGUF
Hermes-2-Pro-Llama-3-8B-GGUF 2024-05-06 8B; Partially censored; JSON, Tool Use, etc. https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF
Llama-3-ChatQA-1.5-8B-GGUF 2024-05-15 8B; Uncensored https://huggingface.co/bartowski/Llama-3-ChatQA-1.5-8B-GGUFF
Hermes-2-Theta-Llama-3-8B-GGUF 2024-05-17 8B; Censored; JSON, Tool Use, etc https://huggingface.co/NousResearch/Hermes-2-Theta-Llama-3-8B-GGUF
blossom-v5.1-9b-GGUF 2024-05-17 9B; Wizard/Orca/Math/Chinese/English mixed dataset https://huggingface.co/bartowski/blossom-v5.1-9b-GGUF
falcon-11B-GGUF 2024-05-17 11B; base raw pretrained model https://huggingface.co/bartowski/falcon-11B-GGUF
llama-3-cat-8b-instruct-v1-GGUFF 2024-05-20 8B; partially censored; system instruction fidelity; Role play https://huggingface.co/bartowski/llama-3-cat-8b-instruct-v1-GGUFF
Yi-1.5-9B-Chat-GGUF 2024-05-20 9B; Censored https://huggingface.co/bartowski/Yi-1.5-9B-Chat-GGUF
SFR-Iterative-DPO-LLaMA-3-8B-R-GGUF 2024-05-22 8B; Partially censored https://huggingface.co/bartowski/SFR-Iterative-DPO-LLaMA-3-8B-R-GGUF
Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-Imatrix 2024-05-22 8B; Roleplay; Trained to not decline requests; Seems to be trained on Quora data https://huggingface.co/Lewdiculous/Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-Imatrix
Mistral-7B-Instruct-v0.3-GGUF 2024-05-23 7B; Censored https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF
L3-8B-Stheno-v3.1-GGUF-IQ-Imatrix 2024-05-30 8B; trained for 1 on 1 uncensored roleplay https://huggingface.co/Lewdiculous/L3-8B-Stheno-v3.1-GGUF-IQ-Imatrix
aya-23-8B-GGUF 2024-05-30 8B; Censored https://huggingface.co/bartowski/aya-23-8B-GGUF
LLaMA3-iterative-DPO-final-GGUF 2024-05-30 8B; Censored https://huggingface.co/bartowski/LLaMA3-iterative-DPO-final-GGUF
openchat-3.6-8b-20240522-GGUF 2024-06-04 8B; Partially censored https://huggingface.co/bartowski/openchat-3.6-8b-20240522-GGUF
Meta-Llama-3-8B-Instruct-abliterated-v3-GGUF 2024-06-04 8B; Uncensored; https://huggingface.co/failspy/Meta-Llama-3-8B-Instruct-abliterated-v3-GGUF
NeuralDaredevil-8B-abliterated-GGUF 2024-06-19 8B; Uncensored; https://huggingface.co/QuantFactory/NeuralDaredevil-8B-abliterated-GGUF
Qwen2-7B-Instruct-GGUF 2024-06-24 7B; Censored https://huggingface.co/Qwen/Qwen2-7B-Instruct-GGUF
DeepSeek-Coder-V2-Lite-Instruct-GGUF 2024-06-27 16B; Instruct coding model https://huggingface.co/lmstudio-community/DeepSeek-Coder-V2-Lite-Instruct-GGUF
internlm2_5-7b-chat-gguf 2024-07-11 7B; Censored; Long context; Reasoning and Tool Use; https://huggingface.co/internlm/internlm2_5-7b-chat-gguf
gemma-2-9b-it-GGUF 2024-07-11 9B; Censored https://huggingface.co/bartowski/gemma-2-9b-it-GGUF
Smegmma-Deluxe-9B-v1-GGUF 2024-07-12 9B; Less censored gemma (still refuses some requests); Role play https://huggingface.co/bartowski/Smegmma-Deluxe-9B-v1-GGUF
h2o-danube3-4b-chat-GGUF 2024-07-17 4B; Partially censored https://huggingface.co/h2oai/h2o-danube3-4b-chat-GGUF
Tiger-Gemma-9B-v1-GGUF 2024-07-17 9B; Uncensored; but appears to have some flaws in logic https://huggingface.co/bartowski/Tiger-Gemma-9B-v1-GGUF
Gemmasutra-9B-v1-GGUF 2024-07-24 9B; Roleplay with some censorship https://huggingface.co/TheDrummer/Gemmasutra-9B-v1-GGUF
Meta-Llama-3.1-8B-Instruct-GGUF 2024-07-25 8B; Censored https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF
Mistral-Nemo-Instruct-2407-GGUF 2024-07-25 12.2B; Partially Uncensored; Recommended by Redditors https://huggingface.co/second-state/Mistral-Nemo-Instruct-2407-GGUF
Celeste-12B-V1.6_iMatrix_GGUF 2024-07-30 12B; Roleplay & story writing model, Uncensored https://huggingface.co/MarsupialAI/Celeste-12B-V1.6_iMatrix_GGUF
Hermes-3-Llama-3.1-8B-GGUF 2024-08-19 8B; Uncensored https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF
Gemma-2-9B-It-SPPO-Iter3-GGUF 2024-08-19 9B; Recommended on Reddit for summarization https://huggingface.co/bartowski/Gemma-2-9B-It-SPPO-Iter3-GGUF
Llama-3.1-Storm-8B-GGUF 2024-08-20 8B; Censored https://huggingface.co/bartowski/Llama-3.1-Storm-8B-GGUF
Phi-3.5-mini-instruct-GGUF 2024-08-21 3.5B; Censored, small & fast https://huggingface.co/lmstudio-community/Phi-3.5-mini-instruct-GGUF
Phi-3.5-mini-instruct_Uncensored-GGUF 2024-08-26 3.5B; Uncensored, small & fast, but has logic flaws & inconsistent results https://huggingface.co/bartowski/Phi-3.5-mini-instruct_Uncensored-GGUF
NemoMix-Unleashed-12B-GGUF 2024-08-26 12B; Partially censored, Roleplaying & story telling https://huggingface.co/bartowski/NemoMix-Unleashed-12B-GGUF
Mistral-NeMo-Minitron-8B-Base-GGUF 2024-08-26 8B; Base model https://huggingface.co/QuantFactory/Mistral-NeMo-Minitron-8B-Base-GGUF
Yi-Coder-9B-Chat-GGUF 2024-09-05 9B; Instruction coding model https://huggingface.co/bartowski/Yi-Coder-9B-Chat-GGUF
Llama-3.2-3B-Instruct-Q8_0-GGUF 2024-10-01 3B; Small censored model https://huggingface.co/hugging-quants/Llama-3.2-3B-Instruct-Q8_0-GGUF
Qwen2.5-7B-Instruct-GGUF 2024-10-01 7B; Censored, Recommended on r/LocalLLaMA https://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF
Qwen2.5-Coder-7B-Instruct-GGUF 2024-10-01 7B; Instruction coding model, recommended on r/LocalLLaMA https://huggingface.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF
Llama-3.1-8B-Lexi-Uncensored-V2-GGUF 2024-10-15 8B; Partially censored https://huggingface.co/Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2-GGUF
SuperNova-Medius-GGUF 2024-10-15 14B; Censored, Recommended on r/LocalLLaMA https://huggingface.co/arcee-ai/SuperNova-Medius-GGUF
Llama-3.2-3B-Instruct-uncensored-GGUF 2024-10-15 3B; Small, uncensored https://huggingface.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-GGUF
Ministral-8B-Instruct-2410-HF-GGUF-TEST 2024-10-21 8B; Partially censored https://huggingface.co/bartowski/Ministral-8B-Instruct-2410-HF-GGUF-TEST
granite-3.0-8b-instruct-GGUF 2024-10-28 8B; Censored https://huggingface.co/bartowski/granite-3.0-8b-instruct-GGUF
aya-expanse-8b-GGUF 2024-10-28 8B; Censored https://huggingface.co/bartowski/aya-expanse-8b-GGUF
Qwen2.5-Coder-14B-Instruct-GGUF 2024-11-12 14B; Coding model, good at its size https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct-GGUF
Marco-o1-GGUF 2024-11-25 7B; CoT model to mimic o1 for reasoning https://huggingface.co/bartowski/Marco-o1-GGUF
Llama-Sentient-3.2-3B-Instruct-GGUF 2024-11-25 3B; Censored https://huggingface.co/prithivMLmods/Llama-Sentient-3.2-3B-Instruct-GGUF
Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-GGUF 2024-12-05 10B; Uncensored story writing model https://huggingface.co/DavidAU/Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-GGUF
INTELLECT-1-Instruct-GGUF 2024-12-05 10B; Censored https://huggingface.co/lmstudio-community/INTELLECT-1-Instruct-GGUF
L3.1-RP-Hero-BigTalker-8B-GGUF 2024-12-05 8B; Uncensored NSFW roleplay model https://huggingface.co/DavidAU/L3.1-RP-Hero-BigTalker-8B-GGUF
EXAONE-3.5-7.8B-Instruct-GGUF 2024-12-09 7.8B; Censored, English and Korean, context up to 32K https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-GGUF
Virtuoso-Small-GGUF 2024-12-10 14B; Censored, good for general tasks, recommended on Reddit https://huggingface.co/bartowski/Virtuoso-Small-GGUF
phi-4 2024-12-16 14B; Censored https://huggingface.co/matteogeniaccio/phi-4
Hermes-3-Llama-3.2-3B-GGUF 2024-12-21 3B; Partially censored https://huggingface.co/NousResearch/Hermes-3-Llama-3.2-3B-GGUF
Falcon3-Mamba-7B-Instruct-GGUF 2024-12-21 7B; Mamba1 architecture, 32K context https://huggingface.co/tiiuae/Falcon3-Mamba-7B-Instruct-GGUF
QwQ-LCoT-7B-Instruct-GGUF 2024-12-21 7B; finetuned on CoT https://huggingface.co/prithivMLmods/QwQ-LCoT-7B-Instruct-GGUF
granite-3.1-8b-instruct-GGUF 2024-12-24 8B; multilingual, 128K context https://huggingface.co/bartowski/granite-3.1-8b-instruct-GGUF
granite-3.1-3b-a800m-instruct-GGUF 2024-12-24 3B; MoE, 128K context https://huggingface.co/bartowski/granite-3.1-3b-a800m-instruct-GGUF
Falcon3-10B-Instruct-GGUF 2025-01-21 10B; 32K context, English/French/Spanish/Portuguese https://huggingface.co/tiiuae/Falcon3-10B-Instruct-GGUF
DeepSeek-R1-Distill-Llama-8B-GGUF 2025-01-21 8B; Reasoning model, 128K context https://huggingface.co/bartowski/DeepSeek-R1-Distill-Llama-8B-GGUF
DeepSeek-R1-Distill-Qwen-7B-GGUF 2025-02-06 7B; Reasoning model, 128K context, censored https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF
DeepSeek-R1-Distill-Qwen-14B-GGUF 2025-02-06 14B; Reasoning model, 128K context, censored https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF
Xwen-7B-Chat-GGUF 2025-02-06 7B; Uncensored https://huggingface.co/mradermacher/Xwen-7B-Chat-GGUF
Virtuoso-Small-v2-GGUF 2025-02-06 14B; Censored https://huggingface.co/bartowski/arcee-ai_Virtuoso-Small-v2-GGUF
DeepHermes-3-Llama-3-8B-Preview-GGUF 2025-02-14 8B; Reasoning model, partially censored https://huggingface.co/NousResearch/DeepHermes-3-Llama-3-8B-Preview-GGUF
Reverb-7b-Q4_K_M-GGUF 2025-02-20 7B; Censored https://huggingface.co/ozone-ai/Reverb-7b-Q4_K_M-GGUF
Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF 2025-03-03 8x3B; MoE, Uncensored NSFW, roleplay, writing https://huggingface.co/DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF
google_gemma-3-12b-it-GGUF 2025-03-17 12B; Censored https://huggingface.co/bartowski/google_gemma-3-12b-it-GGUF
microsoft_Phi-4-mini-instruct-GGUF 2025-03-17 3.8B; Censored, 128 K context https://huggingface.co/bartowski/microsoft_Phi-4-mini-instruct-GGUF
RekaAI_reka-flash-3-GGUF 2025-03-18 21B; Reasoning model, censored https://huggingface.co/bartowski/RekaAI_reka-flash-3-GGUF
mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF 2025-03-19 24B; Censored, unlike previous mistral models https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF
agentica-org_DeepCoder-14B-Preview-GGUF 2025-04-09 14B; Reasoning coding model https://huggingface.co/bartowski/agentica-org_DeepCoder-14B-Preview-GGUF
deepcogito_cogito-v1-preview-qwen-14B-GGUF 2025-04-09 14B; Censored https://huggingface.co/bartowski/deepcogito_cogito-v1-preview-qwen-14B-GGUF
Qwen_Qwen3-14B-GGUF 2025-04-29 14B; Censored https://huggingface.co/bartowski/Qwen_Qwen3-14B-GGUF
ReadyArt_Amoral-Fallen-Omega-Gemma3-12B-GGUF 2025-04-29 12B; Uncensored NSFW roleplay, but may deny non-sexual illegal requests https://huggingface.co/bartowski/ReadyArt_Amoral-Fallen-Omega-Gemma3-12B-GGUF
soob3123_Veritas-12B-GGUF 2025-04-29 12B; Partially censored https://huggingface.co/bartowski/soob3123_Veritas-12B-GGUF
THUDM_GLM-4-9B-0414-GGUF 2025-04-29 9B; Censored https://huggingface.co/bartowski/THUDM_GLM-4-9B-0414-GGUF
microsoft_Phi-4-reasoning-plus-GGUF 2025-05-06 14B; Reasoning model, Censored https://huggingface.co/bartowski/microsoft_Phi-4-reasoning-plus-GGUF
Goekdeniz-Guelmez_Josiefied-Qwen3-8B-abliterated-v1-GGUF 2025-05-06 8B; Uncensored model https://huggingface.co/bartowski/Goekdeniz-Guelmez_Josiefied-Qwen3-8B-abliterated-v1-GGUF
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF 2025-06-09 8B; Reasoning model, Censored https://huggingface.co/bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
open-thoughts_OpenThinker3-7B-GGUF 2025-06-09 7B; Reasoning model, Uncensored, tends to think a lot before answering https://huggingface.co/bartowski/open-thoughts_OpenThinker3-7B-GGUF
Menlo_Jan-nano-128k-GGUF 2025-06-26 4B; Long-context model optimized for MCP use and deep research https://huggingface.co/bartowski/Menlo_Jan-nano-128k-GGUF
google_gemma-3n-E4B-it-GGUF 2025-07-10 8B with 4B activated; Censored, 32K context, efficient parameter management https://huggingface.co/bartowski/google_gemma-3n-E4B-it-GGUF
janhq_Jan-v1-4B-GGUF 2025-08-15 4B; Optimized for search and deep research https://huggingface.co/bartowski/janhq_Jan-v1-4B-GGUF
Qwen3-Coder-30B-A3B-Instruct-GGUF 2025-08-15 30B with 3 B activated; Coding model https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF
Qwen3-30B-A3B-Instruct-2507-GGUF 2025-08-15 30B with 3 B activated; Censored https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF
Qwen3-4B-Instruct-2507-GGUF 2025-08-15 4B; Censored https://huggingface.co/unsloth/Qwen3-4B-Instruct-2507-GGUF
NousResearch_Hermes-4-14B-GGUF 2025-09-09 14B; Partially censored https://huggingface.co/bartowski/NousResearch_Hermes-4-14B-GGUF
VibeThinker-1.5B-GGUFF 2025-11-24 1.5B; Math and Algo Coding model https://huggingface.co/MaziyarPanahi/VibeThinker-1.5B-GGUFF
Ministral-3-14B-Instruct-2512-GGUF 2025-12-08 14B; Multi-modal, Agentic, Censored model https://huggingface.co/unsloth/Ministral-3-14B-Instruct-2512-GGUF
Ministral-3-14B-Reasoning-2512-GGUF 2025-12-08 14B; Reasoning, Multi-modal, Agentic, Censored model https://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-GGUF
NousResearch_NousCoder-14B-GGUF 2026-01-13 14B; Coding model https://huggingface.co/bartowski/NousResearch_NousCoder-14B-GGUF
nvidia_Nemotron-Cascade-14B-Thinking-GGUF 2026-01-13 15B; Coding, reasoning model https://huggingface.co/bartowski/nvidia_Nemotron-Cascade-14B-Thinking-GGUF

Star History

Share This Project

If you find this project helpful, please share it:

Quick Share Links:

You may also be interested in

Check out awesome-TTS-Colab for a curated list of Google Colab notebooks featuring open source text-to-speech (TTS) models. These notebooks are easy to use and can also be run on your local machine!

Older Models with Scores

Older Models with Scores:

These models work better among the models I tested on my hardware (i5-12490F, 32GB RAM, RTX 3060 Ti GDDR6X 8GB VRAM): (Note: Because llama.cpp has made some breaking changes to the support of older ggml models. Some older ggml versions listed below may not work properly on current llama.cpp. But there should be GPTQ equivalents or newer ggml versions for the models.)

NOTE:

  • Major edit on June 30, 2023. Since I noticed GPT-4 started to give more granular scores and tends to give higher scores, I have decided to re-run the tests for all models so they can be compared (All evaluations done with GPT-4-0613 for consistency). The scores in the table below are the average of the scores from the latest tests. The scores in the table below are not comparable to the scores in the previous versions of this README.
Model_Name Avg_Score Colab_Link Date_Added Link
Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) 10.00 2023/10/08 https://huggingface.co/TheBloke/Mistral-7B-OpenOrca-GGUF
Llama-2-13B-chat (using oobabooga/text-generation-webui) 9.65 2023/07/20 https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML
wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) 9.63 2023/05/07 https://huggingface.co/TheBloke/wizard-vicuna-13B-GGML
Nous-Capybara-7B (using oobabooga/text-generation-webui) 9.56 2023/10/08 https://huggingface.co/TheBloke/Nous-Capybara-7B-GGUF
vicuna-13B-v1.5 (using oobabooga/text-generation-webui) 9.53 2023/08/09 https://huggingface.co/TheBloke/vicuna-13B-v1.5-GGML
wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) 9.53 2023/05/29 https://huggingface.co/TheBloke/wizardLM-13B-1.0-GPTQ
airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) 9.50 2023/06/30 https://huggingface.co/TheBloke/airoboros-13B-gpt4-1.4-GPTQ
Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) 9.44 2023/06/03 https://huggingface.co/TheBloke/Nous-Hermes-13B-GPTQ/tree/main
Dolphin-Llama-13B (using oobabooga/text-generation-webui) 9.38 2023/07/24 https://huggingface.co/TheBloke/Dolphin-Llama-13B-GGML
Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) 9.37 2023/10/08 https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF
OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) 9.37 2023/08/15 https://huggingface.co/TheBloke/OpenOrca-Platypus2-13B-GGML
airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) 9.34 2023/08/01 https://huggingface.co/TheBloke/airoboros-l2-13b-gpt4-2.0-GGML
Chronos-13B-v2 (using oobabooga/text-generation-webui) 9.31 2023/08/09 https://huggingface.co/TheBloke/Chronos-13B-v2-GGML
vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) 9.31 2023/06/29 https://huggingface.co/TheBloke/vicuna-13b-v1.3.0-GPTQ
MythoLogic-13B (using oobabooga/text-generation-webui) 9.31 2023/07/20 https://huggingface.co/TheBloke/MythoLogic-13B-GGML
Selfee-13B-GPTQ (using oobabooga/text-generation-webui) 9.28 2023/06/07 https://huggingface.co/TheBloke/Selfee-13B-GPTQ
WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) 9.28 2023/07/26 https://huggingface.co/TheBloke/WizardLM-13B-V1.2-GGML
minotaur-13B-GPTQ (using oobabooga/text-generation-webui) 9.28 2023/06/09 https://huggingface.co/TheBloke/minotaur-13B-GPTQ
Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) 9.20 2023/09/21 https://huggingface.co/TheBloke/Pygmalion-2-13B-SuperCOT2-GGUF
Athena-v1 (using oobabooga/text-generation-webui) 9.19 2023/08/31 https://huggingface.co/TheBloke/Athena-v1-GGUF
PuddleJumper-13B (using oobabooga/text-generation-webui) 9.1875 2023/08/29 https://huggingface.co/TheBloke/PuddleJumper-13B-GGUF
Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) 9.17 2023/07/24 https://huggingface.co/TheBloke/Nous-Hermes-Llama2-GGML
Luban-13B (using oobabooga/text-generation-webui) 9.15 2023/08/31 https://huggingface.co/TheBloke/Luban-13B-GGUF
MythoBoros-13B (using oobabooga/text-generation-webui) 9.15 2023/07/24 https://huggingface.co/TheBloke/MythoBoros-13B-GGML
13B-Ouroboros (using oobabooga/text-generation-webui) 9.11 2023/07/27 https://huggingface.co/TheBloke/13B-Ouroboros-GGML
tulu-13B-GPTQ (using oobabooga/text-generation-webui) 9.06 2023/06/13 https://huggingface.co/TheBloke/tulu-13B-GPTQ
AlpacaCielo-13B (using oobabooga/text-generation-webui) 9.03125 2023/07/27 https://huggingface.co/TheBloke/AlpacaCielo-13B-GGML
StableBeluga-13B (using oobabooga/text-generation-webui) 9 2023/08/01 https://huggingface.co/TheBloke/StableBeluga-13B-GGML
Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) 8.97 2023/08/10 https://huggingface.co/TheBloke/Chronos-Hermes-13B-v2-GGML
OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) 8.97 2023/09/05 https://huggingface.co/TheBloke/OpenBuddy-Llama2-13B-v11.1-GGUF
Camel-Platypus2-13B (using oobabooga/text-generation-webui) 8.94 2023/08/15 https://huggingface.co/TheBloke/Camel-Platypus2-13B-GGML
airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) 8.94 2023/09/21 https://huggingface.co/TheBloke/airoboros-l2-13b-gpt4-m2.0-GGUF
UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) 8.89 2023/06/30 https://huggingface.co/TheBloke/UltraLM-13B-GPTQ
13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) 8.88 2023/06/03 https://huggingface.co/digitous/13B-HyperMantis_GPTQ_4bit-128g/
Stable-Platypus2-13B (using oobabooga/text-generation-webui) 8.875 2023/08/15 https://huggingface.co/TheBloke/Stable-Platypus2-13B-GGML
Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) 8.84 2023/05/25 https://huggingface.co/TheBloke/airoboros-13B-GPTQ
Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) 8.84 2023/09/21 https://huggingface.co/TheBloke/Kuchiki-1.1-L2-7B-GGUF
WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) 8.80625 2023/08/09 https://huggingface.co/TheBloke/WizardLM-1.0-Uncensored-Llama2-13B-GGML
Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) 8.75 2023/08/10 https://huggingface.co/TheBloke/Chronos-Beluga-v2-13B-GGML
Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) 8.75 2023/06/09 https://huggingface.co/TheBloke/Vicuna-13B-CoT-GPTQ
wizardLM-7B.q4_2 (in GPT4All) 8.75 No 2023/05/07 https://gpt4all.io/models/ggml-wizardLM-7B.q4_2.bin
OpenChat_v3.2 (using oobabooga/text-generation-webui) 8.71875 2023/08/01 https://huggingface.co/TheBloke/OpenChat_v3.2-GGML
Huginn-13B (using oobabooga/text-generation-webui) 8.7125 2023/08/10 https://huggingface.co/TheBloke/Huginn-13B-GGML
WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) 8.66 2023/07/17 https://huggingface.co/TheBloke/WizardLM-13B-V1.1-GGML
robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) 8.66 2023/06/19 https://huggingface.co/TheBloke/robin-13B-v2-GPTQ
llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) 8.625 2023/07/21 https://huggingface.co/TheBloke/llama-2-13B-Guanaco-QLoRA-GGML
mpt-7b-chat (in GPT4All) 8.53 No 2023/05/11 https://gpt4all.io/models/ggml-mpt-7b-chat.bin
chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) 8.48125 2023/06/16 https://huggingface.co/TheBloke/chronos-hermes-13B-GPTQ
Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) 8.46875 2023/07/20 https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGML
stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) 8.25 2023/05/12 https://huggingface.co/TheBloke/stable-vicuna-13B-GPTQ
manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) 8.21875 2023/05/24 https://huggingface.co/TheBloke/manticore-13b-chat-pyg-GPTQ
CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) 8.09375 2023/06/10 https://huggingface.co/TheBloke/CAMEL-13B-Combined-Data-GPTQ
WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) 8.09375 2023/06/02 https://huggingface.co/TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ
llama-13b-supercot-GGML (using oobabooga/text-generation-webui) 8.01 2023/07/05 https://huggingface.co/TheBloke/llama-13b-supercot-GGML
Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) 7.96875 2023/05/24 https://huggingface.co/TheBloke/Project-Baize-v2-13B-GPTQ
koala-13B-4bit-128g.GGML (using llama.cpp) 7.9375 No 2023/05/07 https://huggingface.co/TheBloke/koala-13B-GPTQ-4bit-128g-GGML
wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) 7.90625 2023/05/19 https://huggingface.co/4bit/WizardLM-13B-Uncensored-4bit-128g
vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) 7.875 2023/06/29 https://huggingface.co/TheBloke/vicuna-7B-v1.3-GPTQ
Manticore-13B-GPTQ (using oobabooga/text-generation-webui) 7.78125 2023/05/23 https://huggingface.co/TheBloke/Manticore-13B-GPTQ
vicuna-13b-1.1-q4_2 (in GPT4All) 7.75 No 2023/05/07 https://gpt4all.io/models/ggml-vicuna-13b-1.1-q4_2.bin
falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) 7.625 2023/06/02 https://huggingface.co/TheBloke/falcon-7b-instruct-GPTQ
guanaco-13B-GPTQ (using oobabooga/text-generation-webui) 7.5625 2023/05/26 https://huggingface.co/TheBloke/guanaco-13B-GPTQ
Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) 7.31 2023/08/31 https://huggingface.co/TheBloke/Mythical-Destroyer-V2-L2-13B-GGUF
Kimiko-v2-13B (using oobabooga/text-generation-webui) 7.25 2023/08/31 https://huggingface.co/TheBloke/Kimiko-v2-13B-GGUF
orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) 7.0875 2023/06/28 https://huggingface.co/TheBloke/orca_mini_13B-GGML
Platypus2-13B (using oobabooga/text-generation-webui) 7.03125 2023/08/15 https://huggingface.co/TheBloke/Platypus2-13B-GGML
Redmond-Puffin-13B (using oobabooga/text-generation-webui) 7.03125 <
Nearby in the ranking
Similar items