GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
YI
01 ai Yi 34B
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
LL
Llama3-Chinese-8B-Instruct
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
YI
01 ai Yi 6B 200K
Yi · RichardErkhov
Params6.06B
Context195K
Q4_K_M3.7 GB
GR
Granite 20b Code Instruct R1.1
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
GR
Granite 34b Code Instruct 8k
Granite · RichardErkhov
Params33.7B
Context8K
Q4_K_M21.4 GB
YI
Yi 1.5 34B
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
GR
Granite 20b Functioncalling
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
GR
Granite 20b Code Base R1.1
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
GR
Granite 8b Code Base 128k
Granite · RichardErkhov
Params8.05B
Context125K
Q4_K_M4.9 GB
YI
01 ai Yi 6B
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
YI
01 ai Yi 34B Chat
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
GR
Granite 3b Code Instruct
Granite · RichardErkhov
Params3.48B
Context2K
Q4_K_M2.1 GB
GR
Granite 3b Code Base 128k
Granite · RichardErkhov
Params3.48B
Context125K
Q4_K_M2.1 GB
BR
Breeze-7B-Instruct-v0_1
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
SW
Swallow 13b Instruct v0.1
Swallow · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
YI
Yi 1.5 34B Chat 16K
Yi · RichardErkhov
Params34.4B
Context16K
Q4_K_M20.7 GB
LL
Llama-2-7b-WikiChat-fused
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Na0s Llama 3.1 8b Pruned 4 Layers
Llama · RichardErkhov
Params6.94B
Context128K
Q4_K_M4.3 GB
TI
Tinyllama-Cinder-1.3B-Reason-Test
Tinyllama · RichardErkhov
Params1.28B
Context2K
Q4_K_M0.8 GB
MI
Young Children Storyteller Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
Granite 8b Code Instruct 4k
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
MI
Tamil-Mistral-7B-v0.1
Mistral · RichardErkhov
Params7.39B
Context32K
Q4_K_M4.5 GB
LL
chargoddard llama 2 26b trenchcoat stack
Llama · RichardErkhov
Params25.7B
Context—
Q4_K_M5.5 GB
SW
Swallow-70b-NVE-instruct-hf
Swallow · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
TO
tokyotech llm Swallow 7b NVE instruct hf
Tokyotech · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
Mistral 7B Instruct DPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3 Meerkat 8b v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
LI
Lily-Cybersecurity-7B-v0.2
Lily · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SW
Swallow MS 7b Instruct v0.1
Swallow · RichardErkhov
Params7.33B
Context4K
Q4_K_M4.4 GB