GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 Coder 1.5B
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
TI
TinyLlama-1.1B-Chat-v0.5
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
TinyLlama 1.1B Tele It
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
TinyLlama TinyLlama 1.1B Chat v0.3
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
SE
seyf1elislam WestKunai Hermes 10.7b test
Seyf · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
Tool use
BL
Bloom 1b1
Bloom · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
GGUF
MI
Mistral-11B-OmniMix
Mistral · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
OL
OLMo-1B-hf
OLMo · RichardErkhov
Params1.18B
Context2K
Q4_K_M0.7 GB
GGUF
MI
Openbuddy Mistral7B v19.1 4k
Mistral · RichardErkhov
Params7.28B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Amod mental health therapy mistral 7b ins SFT
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
Stablecode Instruct Alpha 3b
Stablecode · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
GGUF
LL
JS CodeLlama 7b 20241203
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QG
qgallouedec tiny GPTNeoXForCausalLM
Qgallouedec · RichardErkhov
Params—
Context—
Q4_K_M0.0 GB
GGUF
MI
Mistral 7B Instruct RDPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
NESPED GEN TinyLlama1B selector and SQL v1 all 16500steps
Nesped · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
SO
Solar Pro Preview Instruct
Solar · RichardErkhov
Params22.1B
Context4K
Q4_K_M13.3 GB
GGUF
CH
ChatWaifu_12B_v2.0
Chatwaifu · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
TI
TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
ML
mlfoundations dev oh dcft v3 sharegpt format sedrick
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Meta Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
KU
Kukedlc Neural 4 ARC 7b
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
YeungNLP firefly qwen1.5 en 7b unsloth
Qwen · RichardErkhov
Params7.72B
Context32K
Q4_K_M1.0 GB
GGUF
EL
Elizezen Berghof ERP 7B
Elizezen · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
GE
gemma-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
UN
UNfilteredAI-1B
Unfilteredai · RichardErkhov
Params1.06B
Context2K
Q4_K_M0.6 GB
GGUF
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
AndreyRzhaksinskiy CDS CodeLlama Ins 7b E2E 20241022 experiment
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Gemma 2 9B
Llama · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
ML
mlabonne BigQwen2.5 Echo 47B Instruct
Mlabonne · RichardErkhov
Params47.4B
Context32K
Q4_K_M28.6 GB
Tool use

Keep browsing