Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
RW
rwkv 6 world 3b
Rwkv · latestissue
Params3.1B
Context1024K
Smallest1.2 GB
GGUF
QW
Qwen3.8-27B
Qwen · k1lliiviie
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
QW
Qwen3.8-27B
Qwen · NilVoid
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
FA
Falcon H1R 7B
Falcon · tiiuae
Params7.59B
Context256K
Q4_K_M4.6 GB
GGUF
QW
Qwen3.8-27B
Qwen · ahmed-amedo
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
QW
Qwen3.8-27B
Qwen · aiartorious
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
MI
Codestral 22B v0.1 IMat
Mistral · legraphista
Params22.2B
Context32K
Smallest4.8 GB
Coding
DE
DeepSeek V2.5
DeepSeek · legraphista
Params236B
Context160K
Smallest47.4 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · CCSSNE
Params34.7B
Context256K
Q4_K_M21.2 GB
Vision
YI
01 ai Yi 34B
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
QW
Qwen3.8-27B
Qwen · haybar4k
Params27.3B
Context256K
Q4_K_M16.6 GB
ReasoningTool useVision
FA
90m Instruct
Falcon · tiiuae
Params91M
Context256K
Q4_K_M0.1 GB
GGUF
QW
Qwen3.8-27B
Qwen · Weirdo333
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
QW
Qwen3.8-27B
Qwen · Aleks199999
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
WH
whisper tiny
Whisper · oxide-lab
Params38M
Context—
Smallest0.0 GB
GGUF
QW
Qwen3.8-27B
Qwen · kcarly
Params27.3B
Context256K
Q4_K_M16.6 GB
ReasoningTool useVision
AN
animagine xl 4.0
Animagine · off-grid-ai
Params3.47B
Context—
Smallest2.8 GB
GGUF
CH
chatterbox
Chatterbox · cstr
Params264M
Context—
Smallest0.0 GB
GGUF
QW
Qwen3.8-27B
Qwen · Fhjdjdbxhshsh
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
LL
Llama3-Chinese-8B-Instruct
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
XL
xLAM 8x7b R
Xlam · legraphista
Params46.7B
Context32K
Smallest9.8 GB
Tool use
LL
Hermes-2-Theta-Llama-3-8B
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
Tool use
QW
Qwen3.8-27B
Qwen · igroc656
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
DA
datagemma rig 27b it IMat
Datagemma · legraphista
Params27.2B
Context8K
Smallest6.1 GB
GGUF
QW
Qwen3.8-27B
Qwen · xrachelburns
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
QW
Qwen3.8-27B
Qwen · Shishkipiski
Params27.3B
Context256K
Q4_K_M16.6 GB
ReasoningTool useVision
YI
01 ai Yi 6B 200K
Yi · RichardErkhov
Params6.06B
Context195K
Q4_K_M3.7 GB
GGUF
QW
Qwen3.8-27B
Qwen · yachiyo8080
Params27.3B
Context256K
Q4_K_M16.6 GB
ReasoningTool useVision
LL
Llama 3.2 3B Instruct
Llama · legraphista
Params3.21B
Context128K
Smallest0.9 GB
GGUF
OR
Ornith 1.5 122B DFlash2
Ornith · kaivoss
Params930M
Context256K
Q4_K_M0.6 GB
GGUF

Other quants

Keep browsing