GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
ST
Opencsg Starcoder2 15b v0.1
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
Coding
SO
SOLAR 10.7B Instruct v1.0 Uncensored
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr5e 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
mirlab AkaLlama llama3 70b v0.1
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
YU
Yuyuan GPT2 110M SciFi Chinese
Yuyuan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
RE
reflect_llm8B_Om2SftT1-Om2G8kOm2Ag40kIpsdpIter1T1_b0.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
RakshitAralimatti Qwen2.5 Coder 1.5B Instruct RASA CALM
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
LL
mlfoundations dev llama3 1 8b mlfoundations dev stackexchange quantumcomputing
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LD
LDCC SOLAR 10.7B
Ldcc · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
GGUF
BL
Bloomz 3b Sft Chat
Bloomz · RichardErkhov
Params3.64B
Context3K
Q4_K_M2.4 GB
GGUF
LL
Llama 3 Patronus Lynx 8B Instruct v1.1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QL
Qllama-.5B-Base-Wiki-Chat-RAG
Qllama · RichardErkhov
Params464M
Context4K
Q4_K_M0.3 GB
GGUF
MI
dolphin-2.6-mistral-7b-dpo-orca-v2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
Deepseek Coder 6.7b Base 20241020
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
TE
tenyx TenyxChat 7B v1
Tenyx · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MI
HanningZhang Mistral MetaMath 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
SA
saishf Fett Eris Mix 7B
Saishf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
Phi 3 Mini 128k Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Speechless Llama2 Luban Orca Platypus 13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
QW
abdulmannan 01 qwen 2.5 0.5b finetuned for sql generation combined dataset
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
PY
Pythia 1b Sft Hh
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Llama3 8b Ko Sft Dpo v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PH
Phi 2 Insurance_Qa Sft Lora
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
GGUF
GE
Gemma 3b 002
Gemma · RichardErkhov
Params3.17B
Context8K
Q4_K_M2.0 GB
GGUF
KA
Katayoon VPO SELM Zephyr 7B iter 2
Katayoon · RichardErkhov
Params7.24B
Context32K
Q4_K_M1.5 GB
GGUF
MI
Infinity Instruct 3M 0625 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama-3-8B-Instruct-v0.4
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
johnpaulbin llama3.1 8b tokipona toki translate 1 fp16
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MA
MagicPrompt-Dalle
Magicprompt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF

Keep browsing