Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
GR
Granite 20b Code Base R1.1
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
CodingReasoning
LL
llava v1.5 7b llamafile
Llama · mozilla-ai
Params6.74B
Context4K
Smallest4.1 GB
Vision
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Unrestricted
Params34.7B
Context256K
Q4_K_M21.2 GB
Vision
QW
qwen3 tts 0.6b base
Qwen · cstr
Params915M
Context—
Smallest0.5 GB
GGUF
VO
voxcpm2
Voxcpm · cstr
Params2.38B
Context—
Smallest0.0 GB
GGUF
GE
gemma-2-9b-it
Gemma · legraphista
Params9.24B
Context8K
Smallest2.4 GB
GGUF
GR
Granite 20b Code Instruct
Granite · legraphista
Params20B
Context8K
Smallest4.5 GB
CodingTool use
GR
Granite 8b Code Base 128k
Granite · RichardErkhov
Params8.05B
Context125K
Q4_K_M4.9 GB
Coding
YI
01 ai Yi 6B
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
YI
01 ai Yi 34B Chat
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · SanctumAI
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GR
Granite 3b Code Instruct
Granite · RichardErkhov
Params3.48B
Context2K
Q4_K_M2.1 GB
Coding
QW
Qwen2-1.5B
Qwen · legraphista
Params1.54B
Context128K
Smallest0.4 GB
GGUF
LL
Llama Guard 3 8B
Llama · legraphista
Params8B
Context128K
Smallest2.0 GB
GGUF
QW
Qwen2-1.5B-Instruct
Qwen · legraphista
Params1.54B
Context32K
Smallest0.4 GB
GGUF
GR
Granite 3b Code Base 128k
Granite · RichardErkhov
Params3.48B
Context125K
Q4_K_M2.1 GB
Coding
BR
Breeze-7B-Instruct-v0_1
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Qwen3.8-27B
Qwen · Hirt556
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
SW
Swallow 13b Instruct v0.1
Swallow · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
GGUF
MI
voxtral mini 4b realtime
Mistral · cstr
Params4.43B
Context—
Smallest2.5 GB
GGUF
RO
RoLlama3-8b-Instruct
Rollama · legraphista
Params8B
Context8K
Smallest2.0 GB
GGUF
PA
Palmyra Fin 70B 32K
Palmyra · legraphista
Params70.6B
Context32K
Smallest15.3 GB
GGUF
QW
Huihui Qwen3.5 122B A10B Abliterated
Qwen · huihui-ai
Params122B
Context256K
Smallest74.2 GB
Vision
MI
mimo asr
Mimo · cstr
Params8.02B
Context—
Smallest0.0 GB
GGUF
FL
FLUX.1 dev
Flux · second-state
Params123M
Context—
Smallest0.1 GB
GGUF
FA
Falcon H1 34B Instruct
Falcon · tiiuae
Params33.6B
Context256K
Q4_K_M20.3 GB
GGUF
LL
saiga_llama3_8b
Llama · IlyaGusev
Params8B
Context8K
Smallest3.2 GB
GGUF
SH
Shieldgemma 2b
Shieldgemma · legraphista
Params2.61B
Context8K
Smallest0.8 GB
GGUF
VI
vibevoice asr
Vibevoice · cstr
Params8.33B
Context—
Smallest0.0 GB
GGUF
GE
gemma 4 e2b yemeni arabic assistant
Gemma · yemeni-ai-lab
Params4.63B
Context128K
Q4_K_M3.4 GB
Vision

Other quants

Keep browsing