GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
JA
jaspionjader Kosmos EVAA PRP v27 8B
Jaspionjader · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
BR
BroAlanTaps GPT2 large 128 28000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
GR
Granite 3.0 3b A800M Instruct
Granite · RichardErkhov
Params3.37B
Context4K
Q4_K_M2.1 GB
Tool use
LL
jkazdan llama8b gsm real sftsd1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 2 Koen 13b
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
LL
ValiantLabs CodeLlama 13B Fireplace
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
LL
Llama3 S Instruct v0.3 Checkpoint 7000 Phase 3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
IN
InternLM2
InternLM · RichardErkhov
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
SO
SOLAR 10.7B Instruct v1.0 128k
Solar · RichardErkhov
Params10.7B
Context128K
Q4_K_M6.5 GB
GGUF
KI
Chat Error Kimiko 10.7B v3
Kimi · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
CodeLlama 13B Python Fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
ML
mlfoundations dev oh teknium scaling down ratiocontrolled 0.3
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
ST
Starling LM 7B beta
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MI
IntelLabs sqft sparsepeft mistral 7b v0.3 60 gsm8k heu
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
LL
smol_llama-4x220M-MoE
Llama · RichardErkhov
Params595M
Context2K
Q4_K_M0.4 GB
GGUF
LL
Llama 3.2 1B Instruct Ja Version2
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
ML
mlfoundations dev oh v1.3 unnatural instructions x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
NE
NESPED GEN TinyLlama schemaLinking min
Nesped · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
aifeifei798 llama3 8B DarkIdol 2.2 Uncensored 1048K
Llama · RichardErkhov
Params8.03B
Context1024K
Q4_K_M4.9 GB
GGUF
SO
SOLAR 10.7B Instruct STOCK_SOLAR Passthrough
Solar · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.0 GB
GGUF
MI
jisukim8873 mistral 7B alpaca case 3 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BL
Bloom 1b7
Bloom · RichardErkhov
Params2.24B
Context4K
Q4_K_M1.5 GB
GGUF
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
DE
Final DEC Deepseek Coder 1.3b Instruct
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
ML
mlfoundations dev oh v1 2 only unnatural instructions
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Hebrew Mistral 7B 200K
Mistral · RichardErkhov
Params7.5B
Context256K
Q4_K_M4.6 GB
GGUF
IN
Instant RP Noodles 12B v1.4
Instant · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
RoleplayTool use
TH
Themis
Themis · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
XA
XALMA 13B Pretrain
Xalma · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
YI
HenryJJ Instruct Yi 6B Dolly CodeAlpaca
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF

Keep browsing