GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen1.5-Wukong-0.5B
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
GGUF
QW
Hi Q krx qwen 2.5 7b it 1104
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GE
Gemma 2 9b
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
PA
padidam gita text generation gpt2
Padidam · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
SM
aryanntated SmolLM 360M text2sql
SmolLM · RichardErkhov
Params362M
Context2K
Q4_K_M0.3 GB
GGUF
CO
Code 290k 6.7B Instruct
Code · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Llama 3.2 1B Instruct SFT D_chosen Pref Mix2
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
DA
DAVinCI Yi Ko 6B v0.61 Ff E1
Davinci · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
TE
TekniumAiroboros-Nebula-7B
Tekniumairoboros · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
Stopwolf Mustra 7B Instruct v0.1
Stopwolf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
osanseviero mistral instruct slerp
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
KA
Karakuri Lm 70b Chat v0.1
Karakuri · RichardErkhov
Params69.2B
Context4K
Q4_K_M41.6 GB
GGUF
LL
shaoleen00 cef llama31 8b 16bits
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
amd AMD Llama 135m code
Llama · RichardErkhov
Params134M
Context2K
Q4_K_M0.1 GB
Coding
DO
DongfuJiang prm gsm 2k with full sol mix ref redistribution hf
Dongfujiang · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama 3.2 1B
Llama · RichardErkhov
Params417M
Context128K
Q4_K_M0.3 GB
GGUF
QW
dat lequoc fast apply 16bit v0.13.2 Qwen2.5 Coder 1.5B
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
LL
Llama 3.2 3b Job Classifier Bot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
TH
TheDrummer Smegmma Deluxe 9B v1
Thedrummer · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
openbuddy-llama3-8b-v21.1-8k
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
RR
rrajvoyxa tinyllama helpdesk lang it v2
Rrajvoyxa · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
QW
QwQ-CoT-0.5B-JA-v0.5
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
ReasoningTool use
GE
lightbird ai gemma 2b merged digital security and privacy
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
MI
Codestral 22B v0.1
Mistral · RichardErkhov
Params22.2B
Context32K
Q4_K_M13.3 GB
Coding
GP
GPT2XL_RLLMv12 Layer 2
Gpt · RichardErkhov
Params1.56B
Context1K
Q4_K_M1.1 GB
GGUF
GE
bitshrine gemma 2 2B hermes function calling
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
Tool use

Keep browsing