GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen1.5-Wukong-0.5B
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
QW
Hi Q krx qwen 2.5 7b it 1104
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GE
Gemma 2 9b
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
PA
padidam gita text generation gpt2
Padidam · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
SM
aryanntated SmolLM 360M text2sql
SmolLM · RichardErkhov
Params362M
Context2K
Q4_K_M0.3 GB
CO
Code 290k 6.7B Instruct
Code · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
Llama 3.2 1B Instruct SFT D_chosen Pref Mix2
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
DA
DAVinCI Yi Ko 6B v0.61 Ff E1
Davinci · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
TE
TekniumAiroboros-Nebula-7B
Tekniumairoboros · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
Stopwolf Mustra 7B Instruct v0.1
Stopwolf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
osanseviero mistral instruct slerp
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
KA
Karakuri Lm 70b Chat v0.1
Karakuri · RichardErkhov
Params69.2B
Context4K
Q4_K_M41.6 GB
LL
shaoleen00 cef llama31 8b 16bits
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
amd AMD Llama 135m code
Llama · RichardErkhov
Params134M
Context2K
Q4_K_M0.1 GB
DO
DongfuJiang prm gsm 2k with full sol mix ref redistribution hf
Dongfujiang · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params417M
Context128K
Q4_K_M0.3 GB
QW
dat lequoc fast apply 16bit v0.13.2 Qwen2.5 Coder 1.5B
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
LL
Llama 3.2 3b Job Classifier Bot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
TH
TheDrummer Smegmma Deluxe 9B v1
Thedrummer · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
LL
openbuddy-llama3-8b-v21.1-8k
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
RR
rrajvoyxa tinyllama helpdesk lang it v2
Rrajvoyxa · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
QW
QwQ-CoT-0.5B-JA-v0.5
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GE
lightbird ai gemma 2b merged digital security and privacy
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
MI
Codestral 22B v0.1
Mistral · RichardErkhov
Params22.2B
Context32K
Q4_K_M13.3 GB
GP
GPT2XL_RLLMv12 Layer 2
Gpt · RichardErkhov
Params1.56B
Context1K
Q4_K_M1.1 GB
GE
bitshrine gemma 2 2B hermes function calling
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB