GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
YI
Yi-1.5-9B-Chat
Yi · RichardErkhov
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
ST
Starling-LM-11B-alpha
Starling · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
LL
Configurable-Llama-3-8B-v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
AN
andykcheng unitrend model 8b vllm
Andykcheng · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
DE
deepseek-coder-1.3b-typescript
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
PE
PersianLLaMA 13B Instruct
Persianllama · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.1 GB
GGUF
CO
CodeQwen1.5 7B Trojan Clas2024 Development
Codeqwen · RichardErkhov
Params7.25B
Context64K
Q4_K_M4.7 GB
GGUF
QW
Josiefied Qwen2.5 1.5B Instruct Abliterated v3
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
YA
yanolja EEVE Korean Instruct 10.8B v1.0
Yanolja · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
DE
DeepSeek V2 Lite Chat
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
GGUF
MI
K2S3 Mistral 7b v1.2
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
LL
Llama3 KALE LM Chem 1.5 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
meta llama CodeLlama 34b Instruct hf
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
Coding
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
K2S3 Mistral 7b v1.0
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
PR
Prometheus 8x7b v2.0
Prometheus · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
NO
nothingiisreal MN 12B Starcannon v2
Nothingiisreal · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
PH
Borea Phi 3.5 Mini Instruct Common
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama2 12.8b_lora Dpo_V1
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
SC
scottsuk0306 zephyr 7b math test
Scottsuk · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
gemma-7b-it
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
LL
ArliAI Llama 3 8B Formax v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LI
lightblue reranker 0.5 cont filt
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
EmbeddingTool use
UN
Unbabel TowerInstruct 7B v0.1
Unbabel · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama3 Med42 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
YBXL Med LLaMA3 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
meta llama Llama 2 7b hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
JI
jieliu Storm 7B
Jieliu · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF

Keep browsing