Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
chrisob94 phi 3 7b dissertation doc chat experiment 10 fold
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
FL
flux 3 image klein base
Flux · leejet
Params5.54B
Context—
Smallest2.3 GB
GGUF
LL
Llama-3-8B-Instruct-64k
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral-7B-Instruct-v0.3
Mistral · SanctumAI
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · legraphista
Params7.62B
Context32K
Smallest1.9 GB
Tool use
OU
outetts 0.3 1b
Outetts · cstr
Params1.19B
Context—
Smallest0.1 GB
GGUF
XL
xLAM 7b R
Xlam · legraphista
Params7.24B
Context32K
Smallest1.6 GB
Tool use
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · legraphista
Params1.54B
Context32K
Smallest0.4 GB
CodingTool use
JP
jpacifico Chocolatine 14B Instruct 4k DPO
Jpacifico · RichardErkhov
Params14B
Context4K
Q4_K_M8.6 GB
GGUF
QW
Qwen2.5 72B Instruct
Qwen · legraphista
Params72.7B
Context32K
Smallest23.7 GB
Tool use
SP
Spark X2.5 4B
Spark · eaddario
Params4.11B
Context—
Smallest0.0 GB
GGUF
CO
confucius4 r2t2
Confucius · cstr
Params2.35B
Context—
Smallest1.5 GB
GGUF
ST
L3-8B-Stheno-v3.2
Stheno · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
NE
addis scribe streaming
Nemotron · addisai
Params99M
Context—
Smallest0.0 GB
GGUF
XL
pcs xlmr base
Xlm · cstr
Params278M
Context—
Smallest0.2 GB
GGUF
LL
Llama 3.1 Minitron 4B Width Base
Llama · legraphista
Params4.51B
Context128K
Smallest1.8 GB
GGUF
MA
Matter 0.1 7B Boost
Matter · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AR
arcee ai Saul Instruct Clown 7b
Arcee · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · Warumauchimmer
Params1.86B
Context—
Smallest0.9 GB
GGUF
LL
Llama 3.1 SuperNova Lite
Llama · legraphista
Params8B
Context128K
Smallest2.0 GB
GGUF
PE
PersianMind v1.0
Persianmind · RichardErkhov
Params6.8B
Context2K
Q4_K_M4.1 GB
GGUF
AY
aya-23-35B
Aya · legraphista
Params35B
Context8K
Smallest8.5 GB
GGUF
GR
grimjim kunoichi lemon royale 7B
Grimjim · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
FL
FlixTest
Flixtest · Huggingbase1baseebasebase
Params27.3B
Context256K
Smallest17.9 GB
GGUF
IR
Irodori-TTS-v4.1-Small
Irodori · sakasegawa
Params859M
Context—
Smallest0.9 GB
GGUF
AU
AutoCoder
Autocoder · legraphista
Params33.3B
Context16K
Smallest7.2 GB
Coding
QW
Qwen3.6 35B A3B
Qwen · eaddario
Params34.7B
Context—
Smallest0.4 GB
GGUF
MO
MOSS Image Base
Moss · leejet
Params5.08B
Context—
Smallest2.2 GB
GGUF
TA
tabcnn
Tabcnn · cstr
Params—
Context—
Smallest0.0 GB
GGUF
PA
parakeet ctc 1.1b ja
Parakeet · cstr
Params1.07B
Context—
Smallest0.7 GB
GGUF

Other quants

Keep browsing