Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
phi 4
Phi · microsoft
Params14.7B
Context16K
Smallest3.6 GB
Coding
OP
openchat-3.6-8b-20240522
Openchat · legraphista
Params8B
Context8K
Smallest2.0 GB
GGUF
DR
dreamshaper xl v2 turbo
Dreamshaper · off-grid-ai
Params3.47B
Context—
Smallest2.8 GB
GGUF
GE
avemio digital gemma 2b Sft pretrain 6000 steps merge
Gemma · RichardErkhov
Params2.5B
Context8K
Q4_K_M1.6 GB
GGUF
VI
vibevoice.cpp models
Vibevoice · mudler
Params—
Context—
Smallest0.0 GB
GGUF
QW
Qwen3.8-27B
Qwen · Greatjedi
Params27B
Context256K
Smallest11.2 GB
GGUF
ST
Step 3.5 Flash
Step · ggml-org
Params197B
Context256K
Smallest118.7 GB
GGUF
GE
Gemma 2 27b It
Gemma · legraphista
Params27.2B
Context8K
Smallest6.1 GB
GGUF
NE
Huihui NeoHorse 1 4B Abliterated MTP
Neohorse · huihui-ai
Params4.33B
Context256K
Smallest2.3 GB
ReasoningTool useVision
QW
Qwen3.6-27B-Uncensored-HauhauCS-Aggressive
Qwen · SummonGovernance
Params27.3B
Context256K
Smallest18.0 GB
GGUF
HU
Huihui4 8B A4B
Huihui · huihui-ai
Params8.1B
Context256K
Smallest5.4 GB
Vision
PH
gretelai Phi 3 mini 128k instruct
Phi · RichardErkhov
Params1.9B
Context—
Q4_K_M0.0 GB
GGUF
ID
ideogram 4
Ideogram · stduhpf
Params9.28B
Context—
Smallest3.6 GB
GGUF
QW
qwen3 asr 1.7b
Qwen · cstr
Params2.35B
Context—
Smallest1.5 GB
GGUF
QW
FLUX 3 Image Klein TE Uncensored
Qwen · AtomicChat
Params4.02B
Context256K
Smallest2.5 GB
CodingTool useVision
LL
tachyphylaxis Meta Llama 3.1 Chat Uncensored
Llama · RichardErkhov
Params15M
Context—
Q4_K_M0.0 GB
GGUF
WA
NSFW Wan 14b
Wan · NSFW-API
Params14.3B
Context—
Smallest9.7 GB
GGUF
LL
KISTI KONI KONI Llama3 8B Instruct 20240729
Llama · RichardErkhov
Params8B
Context—
Q4_K_M0.0 GB
GGUF
MO
Moody Krea 2 Mix V4
Moody · Henrysk
Params12.8B
Context—
Smallest2.5 GB
GGUF
PH
Phi 3 mini 4k instruct update2024 07 03 IMat
Phi · legraphista
Params3.82B
Context4K
Smallest0.8 GB
Coding
TE
test gguf moe sample
Test · SzymonOzog
Params403M
Context—
Smallest0.1 GB
GGUF
LL
ggml llava v1.5 7b
LLaVA · mys
Params6.74B
Context4K
Smallest4.1 GB
Vision
NE
Nemotron3-Nano-4B-Uncensored-HauhauCS-Aggressive
Nemotron · HauhauCS
Params3.97B
Context1024K
Q4_K_M2.8 GB
GGUF
GL
GLM-5.3-Flash
GLM · ggml-org
Params313B
Context1024K
Smallest4.9 GB
Vision
ID
ideogram 4
Ideogram · rectangleworm
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
CO
Yi Coder 9B Chat
Coder · RichardErkhov
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
QW
Qwen3-TTS-12Hz-1.7B-CustomVoice
Qwen · sakasegawa
Params2.03B
Context—
Smallest1.3 GB
GGUF
LL
SageV2 8B LongContext NSFW RP
Llama · sinhal
Params8.03B
Context128K
Smallest3.2 GB
Roleplay
SP
Spark X2.5 4B Abliterated
Spark · SC117
Params4.11B
Context1024K
Smallest2.4 GB
GGUF
MI
maywell Mistral ko 7B v0.1
Mistral · RichardErkhov
Params1.5B
Context—
Q4_K_M0.0 GB
GGUF

Other quants

Keep browsing