GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
IN
interview eval zephyr 7b stem case 7
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
LL
verbalyze Llama3.1 finetuned
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
RE
reverie-7b
Reverie · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
ministral Ministral 3b instruct
Mistral · RichardErkhov
Params3.32B
Context128K
Q4_K_M2.0 GB
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
LL
mlfoundations dev llama3 1 8b oh dcft v1 wo camel biology gpt 4o mini
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
CodeLlama 34B Python Fp16
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
ML
mlfoundations dev oh teknium scaling down random 0.1
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama Guard 3 1B
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
DB
Dbrx Instruct
Dbrx · RichardErkhov
Params208B
Context32K
Q4_K_M79.9 GB
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
SA
SambaLingo Russian Chat
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
GP
gpt2-medium-italian-embeddings
Gpt · RichardErkhov
Params365M
Context1K
Q4_K_M0.2 GB
MI
Oh Mistral bs4096_lr5_00E 06_schedulerconstant_warmup1_00E 01_minlr
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
CH
Chatgpt2
Chatgpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GR
Granite Guardian 3.0 8b
Granite · RichardErkhov
Params8.17B
Context8K
Q4_K_M4.9 GB
ST
Starcoder2 15b
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
TI
Tinyllama_Alpaca_Sft_Sample
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
BI
bitnet_b1_58-large
Bitnet · RichardErkhov
Params729M
Context2K
Q4_K_M0.5 GB
SM
SmolLM 135M Instruct
SmolLM · RichardErkhov
Params135M
Context2K
Q4_K_M0.1 GB
LL
Llama 2 7b chat_FFT_Alpaca Gpt4 Zh
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
PH
dbands Phi 3.5 mini instruct pyomo pysim coder
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
LL
Llama 3.2 3B Instruct SFT D_chosen Pref Mix
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GE
vaniebermudez gemma 2b instruct ft squad qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB