GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
IN
interview eval zephyr 7b stem case 7
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
LL
verbalyze Llama3.1 finetuned
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
RE
reverie-7b
Reverie · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
ministral Ministral 3b instruct
Mistral · RichardErkhov
Params3.32B
Context128K
Q4_K_M2.0 GB
GGUF
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh dcft v1 wo camel biology gpt 4o mini
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
CodeLlama 34B Python Fp16
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
Coding
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
ML
mlfoundations dev oh teknium scaling down random 0.1
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama Guard 3 1B
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
GGUF
DB
Dbrx Instruct
Dbrx · RichardErkhov
Params208B
Context32K
Q4_K_M79.9 GB
GGUF
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
SA
SambaLingo Russian Chat
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
GGUF
GP
gpt2-medium-italian-embeddings
Gpt · RichardErkhov
Params365M
Context1K
Q4_K_M0.2 GB
Embedding
MI
Oh Mistral bs4096_lr5_00E 06_schedulerconstant_warmup1_00E 01_minlr
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
CH
Chatgpt2
Chatgpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
GR
Granite Guardian 3.0 8b
Granite · RichardErkhov
Params8.17B
Context8K
Q4_K_M4.9 GB
GGUF
ST
Starcoder2 15b
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
Coding
TI
Tinyllama_Alpaca_Sft_Sample
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
BI
bitnet_b1_58-large
Bitnet · RichardErkhov
Params729M
Context2K
Q4_K_M0.5 GB
GGUF
SM
SmolLM 135M Instruct
SmolLM · RichardErkhov
Params135M
Context2K
Q4_K_M0.1 GB
GGUF
LL
Llama 2 7b chat_FFT_Alpaca Gpt4 Zh
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
PH
dbands Phi 3.5 mini instruct pyomo pysim coder
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
Coding
LL
Llama 3.2 3B Instruct SFT D_chosen Pref Mix
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
GE
vaniebermudez gemma 2b instruct ft squad qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF

Keep browsing