Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
Meta-Llama-3-8B-Instruct-abliterated-v3
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
GR
Granite 34b Code Instruct 8k
Granite · RichardErkhov
Params33.7B
Context8K
Q4_K_M21.4 GB
LL
dolphin-2.9.1-llama-3-8b
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
WH
whisper large v3 turbo
Whisper · oxide-lab
Params809M
Context—
Smallest0.3 GB
CO
codegeex4-all-9b
Codegeex · legraphista
Params9.4B
Context128K
Smallest3.1 GB
GE
Vikhr-Gemma-2B-instruct
Gemma · Vikhrmodels
Params2.61B
Context8K
Q4_K_M1.7 GB
YI
Yi 1.5 34B
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
GR
Granite 20b Functioncalling
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
GR
Granite 20b Code Base R1.1
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Unrestricted
Params34.7B
Context256K
Q4_K_M21.2 GB
GE
gemma-2-9b-it
Gemma · legraphista
Params9.24B
Context8K
Smallest2.4 GB
GR
Granite 20b Code Instruct
Granite · legraphista
Params20B
Context8K
Smallest4.5 GB
GR
Granite 8b Code Base 128k
Granite · RichardErkhov
Params8.05B
Context125K
Q4_K_M4.9 GB
YI
01 ai Yi 6B
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
YI
01 ai Yi 34B Chat
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
LL
Meta Llama 3.1 8B Instruct
Llama · SanctumAI
Params8B
Context128K
Q4_K_M4.9 GB
GR
Granite 3b Code Instruct
Granite · RichardErkhov
Params3.48B
Context2K
Q4_K_M2.1 GB
QW
Qwen2-1.5B
Qwen · legraphista
Params1.54B
Context128K
Smallest0.4 GB
LL
Llama Guard 3 8B
Llama · legraphista
Params8B
Context128K
Smallest2.0 GB
QW
Qwen2-1.5B-Instruct
Qwen · legraphista
Params1.54B
Context32K
Smallest0.4 GB
GR
Granite 3b Code Base 128k
Granite · RichardErkhov
Params3.48B
Context125K
Q4_K_M2.1 GB
BR
Breeze-7B-Instruct-v0_1
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
QW
Qwen3.8-27B
Qwen · Hirt556
Params27.3B
Context256K
Q4_K_M16.6 GB
SW
Swallow 13b Instruct v0.1
Swallow · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
RO
RoLlama3-8b-Instruct
Rollama · legraphista
Params8B
Context8K
Smallest2.0 GB
PA
Palmyra Fin 70B 32K
Palmyra · legraphista
Params70.6B
Context32K
Smallest15.3 GB
LL
llava 1.6
LLaVA · cmp-nct
Params7.24B
Context32K
Smallest4.4 GB
FA
Falcon H1 34B Instruct
Falcon · tiiuae
Params33.6B
Context256K
Q4_K_M20.3 GB
SH
Shieldgemma 2b
Shieldgemma · legraphista
Params2.61B
Context8K
Smallest0.8 GB
GE
gemma 4 e2b yemeni arabic assistant
Gemma · yemeni-ai-lab
Params4.63B
Context128K
Q4_K_M3.4 GB