GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
YI
01 ai Yi 34B
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
LL
Llama3-Chinese-8B-Instruct
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
YI
01 ai Yi 6B 200K
Yi · RichardErkhov
Params6.06B
Context195K
Q4_K_M3.7 GB
GGUF
GR
Granite 20b Code Instruct R1.1
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
CodingReasoning
GR
Granite 34b Code Instruct 8k
Granite · RichardErkhov
Params33.7B
Context8K
Q4_K_M21.4 GB
Coding
YI
Yi 1.5 34B
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
GR
Granite 20b Functioncalling
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
GGUF
GR
Granite 20b Code Base R1.1
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
CodingReasoning
GR
Granite 8b Code Base 128k
Granite · RichardErkhov
Params8.05B
Context125K
Q4_K_M4.9 GB
Coding
YI
01 ai Yi 6B
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
YI
01 ai Yi 34B Chat
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
GR
Granite 3b Code Instruct
Granite · RichardErkhov
Params3.48B
Context2K
Q4_K_M2.1 GB
Coding
GR
Granite 3b Code Base 128k
Granite · RichardErkhov
Params3.48B
Context125K
Q4_K_M2.1 GB
Coding
BR
Breeze-7B-Instruct-v0_1
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
GGUF
SW
Swallow 13b Instruct v0.1
Swallow · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
GGUF
YI
Yi 1.5 34B Chat 16K
Yi · RichardErkhov
Params34.4B
Context16K
Q4_K_M20.7 GB
GGUF
LL
Llama-2-7b-WikiChat-fused
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Na0s Llama 3.1 8b Pruned 4 Layers
Llama · RichardErkhov
Params6.94B
Context128K
Q4_K_M4.3 GB
GGUF
TI
Tinyllama-Cinder-1.3B-Reason-Test
Tinyllama · RichardErkhov
Params1.28B
Context2K
Q4_K_M0.8 GB
GGUF
MI
Young Children Storyteller Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
Granite 8b Code Instruct 4k
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
Coding
MI
Tamil-Mistral-7B-v0.1
Mistral · RichardErkhov
Params7.39B
Context32K
Q4_K_M4.5 GB
GGUF
LL
chargoddard llama 2 26b trenchcoat stack
Llama · RichardErkhov
Params25.7B
Context—
Q4_K_M5.5 GB
GGUF
SW
Swallow-70b-NVE-instruct-hf
Swallow · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
TO
tokyotech llm Swallow 7b NVE instruct hf
Tokyotech · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B Instruct DPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 Meerkat 8b v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LI
Lily-Cybersecurity-7B-v0.2
Lily · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SW
Swallow MS 7b Instruct v0.1
Swallow · RichardErkhov
Params7.33B
Context4K
Q4_K_M4.4 GB
GGUF

Keep browsing