GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
nbeerbower mistral nemo gutenberg 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
MI
Radintloom Mistral 7b Fusion
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AI
AIdenU SOLAR 10.7b ko Y24 v0.1
Aidenu · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
DE
KSU HW SEC Deepseek SFT RL AST only
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
LL
lotfsadek llama3.1 finetuned py v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Weyaxi MetaMath NeuralHermes 2.5 Mistral 7B Linear
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
PH
Ganidu Phi 3 mini 4k OLScience v02
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
BL
Blexus Quble test model v1 INSTRUCT v1
Blexus · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
GE
SauerkrautLM-Gemma-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
SA
saishf Ortho SOVL 8B L3
Saishf · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
BL
Bloom 560m
Bloom · RichardErkhov
Params816M
Context1K
Q4_K_M0.6 GB
GGUF
BR
BroAlanTaps GPT2 large 256 15000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
NA
Narumashi-RT-11B
Narumashi · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
AE
aetherwiing MN 12B Starcannon v1
Aetherwiing · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
MI
CorticalStack mistral 7b tak stack dpo
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TA
taicheng zephyr 7b align scan 0.0 0.7 cosine 3
Taicheng · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MA
Maestrale Chat v0.3 Beta
Maestrale · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2-0.5B-Chat_SFT_DPO
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
QW
bunnycore Qwen2.5 1.5B Matrix
Qwen · RichardErkhov
Params1.78B
Context32K
Q4_K_M1.1 GB
Tool use
BL
Bloom 1b1
Bloom · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
GGUF
NO
Nova 13B 50 Step
Nova · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
mistral_2X7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
DA
datatab Yugo55A GPT
Datatab · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
OpenBuddy openbuddy llama3.1 8b v22.3 131k
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
incisolr gemma 2b instruct ft datascienceinterview qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
UCLA AGI Llama 3 Instruct 8B SPPO Iter3
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Keep browsing