GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
MinaMila GermanCredit Ext Mitig CFA llama 3 InstBase ImpFeat 20ep withsex
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
Qwen2.5 1.5B Instruct MLX
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
LL
grimjim Llama 3 Oasis v1 OAS 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
BL
Bloomz 3b
Bloomz · RichardErkhov
Params3.64B
Context2K
Q4_K_M2.4 GB
GGUF
QW
Qwen2 0.5B Instruct
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
GW
Gweizheng Marcoro14 7B dare
Gweizheng · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GH
Ghost 8b Beta 1608
Ghost · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
LL
Llama 7b Hf
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
GE
google-gemma-2b-it-dequantized
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler linear warmup0.10 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BA
Balaaditya GPT2 Finetuned Emotion Classification
Balaaditya · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
LL
mlfoundations dev llama3 1 8b mlfoundations dev stackoverflow 25000tasks 75p
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
FI
Fixed_Sana
Fixed · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
QW
dat lequoc fast apply 16bit v0.v15 Qwen2.5 Coder 1.5B
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
ST
StableCode 3B
Stablecode · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
GGUF
AU
automerger ShadowYamshadow 7B
Automerger · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
NeuralExperiment 7b MagicCoder v7
Neuralexperiment · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
MI
unsloth Hermes 2 Pro Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
TI
TinyLlama 1.1B Tele
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
RU
ruchi012 saul 7B merged 3000
Ruchi · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LU
lumatic ai BongLlama 1.1B Chat alpha v0
Lumatic · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
MI
Mistral-7B-Merge-14-v0.5
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Embedding
PH
Phi 3 Medium 4k Instruct
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.4 GB
GGUF
MI
NeuralHermes-2.5-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
jan hq llama3 s instruct v0.3 checkpoint 6000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
JINJIN7987 llama3 3b neg mtba
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
LL
popatkishan1 Llama 3 8B Instruct AIH RAG
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ZU
a1zubair RedditData Fine Tuned GPT2
Zubair · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF

Keep browsing