GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 14B Instruct
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool use
UN
Unbabel TowerInstruct 13B v0.1
Unbabel · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
Meta Llama 3.1 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
mlfoundations dev hp ablations grid mistral bsz1024 lr2e 6 scheduler cosine warmup0.15
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
LION LLaMA 3 8b Dpo v1.0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler linear warmup0.10
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
bunnycore Phi 2 OmniMatrix
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
GGUF
LL
codellama CodeLlama 34b Instruct hf
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
Coding
GE
Gemma 2 9b It 390K
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
LL
ihughes15234 llama 3 1 8bi tictactoe dpo1epoch v3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
kayfour T3Q llama3.1 8B it Ko safe
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
MH
MHGanainy gpt2 xl lora multi shared 512 top
Mhganainy · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
ME
Merdeka LLM merdeka llm lawyer 3b 128k instruct
Merdeka · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
OP
Opencsg Stable Code 3b v1
Opencsg · RichardErkhov
Params2.8B
Context16K
Q4_K_M1.7 GB
Coding
MA
martimfasantos tinyllama 1.1b mt sft full
Martimfasantos · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Llama 3.2 3B Wildguard Ko 2410
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
LL
ValiantLabs Llama2 13B ShiningValiant
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
PY
pythia-160m_ft_CookingRecipes
Pythia · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
GGUF
GR
Grafted Hermetic Platypus B 2x7B
Grafted · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
MI
Chinese Mistral 7B v0.1
Mistral · RichardErkhov
Params7.5B
Context32K
Q4_K_M4.5 GB
GGUF
XU
XuanYuan 6B
Xuanyuan · RichardErkhov
Params6.39B
Context8K
Q4_K_M3.9 GB
GGUF
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
LL
smol_llama-220M-open_instruct
Llama · RichardErkhov
Params218M
Context2K
Q4_K_M0.1 GB
GGUF
YI
Yi 34B 200K DARE Merge v5
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
LL
Suzume Llama 3 8B Multilingual
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SO
I SOLAR 10.7B Sft v1.0
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
HA
habanoz tinyllama oasst1 top1 instruct full lr1 5 v0.1
Habanoz · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF

Keep browsing