GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
JA
jaspionjader Kosmos EVAA PRP v27 8B
Jaspionjader · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
BR
BroAlanTaps GPT2 large 128 28000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GR
Granite 3.0 3b A800M Instruct
Granite · RichardErkhov
Params3.37B
Context4K
Q4_K_M2.1 GB
LL
jkazdan llama8b gsm real sftsd1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Llama 2 Koen 13b
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
LL
ValiantLabs CodeLlama 13B Fireplace
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
LL
Llama3 S Instruct v0.3 Checkpoint 7000 Phase 3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
IN
InternLM2
InternLM · RichardErkhov
Params7.74B
Context8K
Q4_K_M4.7 GB
SO
SOLAR 10.7B Instruct v1.0 128k
Solar · RichardErkhov
Params10.7B
Context128K
Q4_K_M6.5 GB
KI
Chat Error Kimiko 10.7B v3
Kimi · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
LL
CodeLlama 13B Python Fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
ML
mlfoundations dev oh teknium scaling down ratiocontrolled 0.3
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
ST
Starling LM 7B beta
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
MI
IntelLabs sqft sparsepeft mistral 7b v0.3 60 gsm8k heu
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
LL
smol_llama-4x220M-MoE
Llama · RichardErkhov
Params595M
Context2K
Q4_K_M0.4 GB
LL
Llama 3.2 1B Instruct Ja Version2
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
ML
mlfoundations dev oh v1.3 unnatural instructions x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
NE
NESPED GEN TinyLlama schemaLinking min
Nesped · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
aifeifei798 llama3 8B DarkIdol 2.2 Uncensored 1048K
Llama · RichardErkhov
Params8.03B
Context1024K
Q4_K_M4.9 GB
SO
SOLAR 10.7B Instruct STOCK_SOLAR Passthrough
Solar · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.0 GB
MI
jisukim8873 mistral 7B alpaca case 3 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
BL
Bloom 1b7
Bloom · RichardErkhov
Params2.24B
Context4K
Q4_K_M1.5 GB
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
DE
Final DEC Deepseek Coder 1.3b Instruct
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
ML
mlfoundations dev oh v1 2 only unnatural instructions
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Hebrew Mistral 7B 200K
Mistral · RichardErkhov
Params7.5B
Context256K
Q4_K_M4.6 GB
IN
Instant RP Noodles 12B v1.4
Instant · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
TH
Themis
Themis · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
XA
XALMA 13B Pretrain
Xalma · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
YI
HenryJJ Instruct Yi 6B Dolly CodeAlpaca
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB