Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LA
Laguna-S-2.1
Laguna · huihui-ai
Params118B
Context256K
Smallest53.2 GB
GGUF
OR
Ornith 1.5 35B A3B
Ornith · AtomicChat
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
GL
GLM 4.7 Flash
GLM · ggml-org
Params30B
Context198K
Smallest18.2 GB
GGUF
GR
Grok 1
Grok · Arki05
Params316B
Context8K
Smallest65.3 GB
GGUF
SA
Sao10K L3 8B Stheno v3.1
Sao · RichardErkhov
Params8B
Context—
Q4_K_M0.3 GB
GGUF
LL
Llama Joycaption Beta One Hf Llava
Llama · concedo
Params8B
Context128K
Smallest4.9 GB
Vision
MI
avemio digital mistral 7b Sft pretrain 6000 steps merge
Mistral · RichardErkhov
Params7.24B
Context—
Q4_K_M0.1 GB
GGUF
MI
MiniCPM-V-4.7-35B-A3B
MiniCPM · AtomicChat
Params34.7B
Context256K
Smallest10.1 GB
ReasoningTool useVision
FL
FLUX.1 schnell
Flux · leejet
Params11.9B
Context—
Smallest4.1 GB
GGUF
MI
Ming 3.0 Audio Tiny
Ming · audio-cpp
Params—
Context—
Smallest10.0 GB
GGUF
FL
FLUX 3 Image Klein
Flux · AtomicChat
Params5.55B
Context256K
Smallest2.5 GB
GGUF
LL
testmod koen llama2 13b dpotrain testver
Llama · RichardErkhov
Params13B
Context—
Q4_K_M0.0 GB
GGUF
QW
Qwen3.6 35B A3B Uncensored HauhauCS MTP
Qwen · morikomorizz
Params447M
Context256K
Smallest15.9 GB
Vision
BR
breeze-tts-2
Breeze · cstr
Params2.85B
Context—
Smallest2.2 GB
GGUF
BA
BarryFutureman NeuralTurdusVariant1 7B
Barryfutureman · RichardErkhov
Params7B
Context—
Q4_K_M0.0 GB
GGUF
OR
Ornith-1.5-122B
Ornith · AtomicChat
Params125B
Context256K
Smallest35.0 GB
ReasoningTool useVision
MO
MOSS Image SFT
Moss · leejet
Params5.08B
Context—
Smallest2.2 GB
GGUF
TE
Test Frankenstein
Test · ND911
Params11.9B
Context—
Smallest6.9 GB
GGUF
LL
VAGOsolutions Llama 3 SauerkrautLM 70b Instruct
Llama · RichardErkhov
Params6B
Context—
Q4_K_M0.0 GB
GGUF
DR
Drama 3 Windows
Drama · Eng-Ahmd
Params35.9B
Context—
Smallest1.6 GB
GGUF
UN
Undi95 Emerhyst 20B
Undi · RichardErkhov
Params920M
Context—
Q4_K_M0.0 GB
GGUF
WH
whisper large v3 turbo
Whisper · xkeyC
Params809M
Context—
Smallest0.5 GB
GGUF
MI
minmax
Minmax · Kutches
Params20.1B
Context—
Q4_K_M11.5 GB
GGUF
LL
4yo1 llama3 pre1 pre2 ds ins2 lora3
Llama · RichardErkhov
Params200M
Context—
Q4_K_M0.0 GB
GGUF
DE
DeepSeek-V2-Lite
DeepSeek · legraphista
Params15.7B
Context160K
Smallest5.0 GB
GGUF
MI
MiniCPM V 2 6
MiniCPM · openbmb
Params7.61B
Context32K
Q4_K_M4.7 GB
Vision
FU
Fun-CosyVoice3-0.5B-2512-voices
Fun · cstr
Params7M
Context—
Smallest0.0 GB
GGUF
FL
PornMaster Klein 9b
Flux · rectangleworm
Params8.19B
Context256K
Q4_K_M5.0 GB
GGUF
ZO
ZONOS2
Zonos · Zyphra
Params64M
Context—
Smallest0.0 GB
GGUF
GE
Gemmasutra Mini 2B v1
Gemma · TheDrummer
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF

Other quants

Keep browsing