Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
h2oai h2ogpt 16k codellama 13b instruct
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M1.7 GB
Coding
QW
Karlsen Chat 27B abliterated
Qwen · karlsencoin
Params27.3B
Context256K
Q4_K_M16.6 GB
ReasoningTool useVision
ST
starcoder2-15b
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
Coding
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · hardcoremoore
Params1.86B
Context—
Smallest0.9 GB
GGUF
BA
bark-small
Bark · cstr
Params404M
Context—
Smallest0.2 GB
GGUF
JS
jsfs11 MixtureofMerges MoE 2x7b v7
Jsfs · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
LL
OpenMath2 Llama3.1 8B
Llama · RichardErkhov
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
ZhangShenao SELM Llama 3 8B Instruct iter 3
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
WA
wav2vec2 large xlsr 53 chinese zh cn
Wav · cstr
Params319M
Context—
Smallest0.2 GB
GGUF
QR
Qra 7b Dolly Instruction 0.1
Qra · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
DE
DeepSeek Coder V2 Instruct 0724
DeepSeek · legraphista
Params236B
Context160K
Smallest47.4 GB
CodingTool use
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · LxBarn
Params1.86B
Context256K
Smallest0.9 GB
GGUF
ME
Meltemi 7B Instruct v1.5
Meltemi · RichardErkhov
Params7.48B
Context32K
Q4_K_M4.5 GB
GGUF
MI
chihoonlee10 T3Q EN DPO Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FA
Falcon H1 3B Instruct
Falcon · tiiuae
Params3.15B
Context128K
Q4_K_M1.9 GB
GGUF
CO
Yi Coder 9B Chat
Coder · legraphista
Params8.83B
Context128K
Smallest2.0 GB
Coding
PA
parakeet tdt ctc 1.1b
Parakeet · cstr
Params1.07B
Context—
Smallest0.7 GB
GGUF
CO
CodeScout 14B
Codescout · n0ni
Params14.8B
Context40K
Q4_K_M9.0 GB
ReasoningTool use
PH
dolphin-2.9.2-Phi-3-Medium-abliterated
Phi · legraphista
Params14B
Context4K
Smallest3.1 GB
CodingTool use
RE
reader lm 1.5b IMat
Reader · legraphista
Params1.54B
Context250K
Smallest0.4 GB
GGUF
NE
neo_7b_sft_v0.1
Neo · legraphista
Params7.79B
Context8K
Smallest1.7 GB
GGUF
RO
RORem mixed
Rorem · mayocream
Params2.57B
Context—
Smallest2.0 GB
GGUF
IC
icefog72 IceLemonTeaRP 32k 7b
Icefog · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FI
FireRedLID
Fireredlid · cstr
Params887M
Context—
Smallest0.4 GB
GGUF
GE
Vikhr-Gemma-2B-instruct
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
WH
whisper small
Whisper · oxide-lab
Params242M
Context—
Smallest0.1 GB
GGUF
GO
Google_Medgemma 1.5 4b It
Google · Bercraft
Params3.88B
Context128K
Q4_K_M2.5 GB
ReasoningVision
FO
FoxEngineAi Mega Destroyer 8x7B
Foxengineai · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
QW
Qwen2.5 32B Instruct
Qwen · legraphista
Params32.8B
Context32K
Smallest7.3 GB
Tool use
QW
Qwen2.5 Math 7B Instruct
Qwen · legraphista
Params7.62B
Context4K
Smallest1.9 GB
Tool use

Other quants

Keep browsing