GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
YI
Yi-1.5-9B-Chat
Yi · RichardErkhov
Params8.83B
Context4K
Q4_K_M5.3 GB
ST
Starling-LM-11B-alpha
Starling · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
LL
Configurable-Llama-3-8B-v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
AN
andykcheng unitrend model 8b vllm
Andykcheng · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DE
deepseek-coder-1.3b-typescript
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
PE
PersianLLaMA 13B Instruct
Persianllama · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.1 GB
CO
CodeQwen1.5 7B Trojan Clas2024 Development
Codeqwen · RichardErkhov
Params7.25B
Context64K
Q4_K_M4.7 GB
QW
Josiefied Qwen2.5 1.5B Instruct Abliterated v3
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
YA
yanolja EEVE Korean Instruct 10.8B v1.0
Yanolja · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
DE
DeepSeek V2 Lite Chat
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
MI
K2S3 Mistral 7b v1.2
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
LL
Llama3 KALE LM Chem 1.5 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
meta llama CodeLlama 34b Instruct hf
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
K2S3 Mistral 7b v1.0
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
PR
Prometheus 8x7b v2.0
Prometheus · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
NO
nothingiisreal MN 12B Starcannon v2
Nothingiisreal · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
PH
Borea Phi 3.5 Mini Instruct Common
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
LL
Llama2 12.8b_lora Dpo_V1
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
SC
scottsuk0306 zephyr 7b math test
Scottsuk · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
gemma-7b-it
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
LL
ArliAI Llama 3 8B Formax v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
LI
lightblue reranker 0.5 cont filt
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
UN
Unbabel TowerInstruct 7B v0.1
Unbabel · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
Llama3 Med42 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
YBXL Med LLaMA3 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
LL
meta llama Llama 2 7b hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
JI
jieliu Storm 7B
Jieliu · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB