Q4_K_M GGUF models

4-bit k-quant, medium. The common default: a good balance of size and quality. 148631 models have a Q4_K_M file. Sizes run from 1M to 2450B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.8-27B
Qwen · k1lliiviie
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
CE
MN 12B Celeste V1.9
Celeste · bartowski
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
OrionStar-Yi-34B-Chat-Llama
Llama · mradermacher
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
CY
Cydonia 24B v4.2.0
Cydonia · bartowski
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
MI
Mistral Small 24B Instruct 2501
Mistral · bartowski
Params23.6B
Context32K
Q4_K_M14.3 GB
GGUF
ER
10Eros Max Hybrid Beta5
Eros · Abiray
Params20.1B
Context—
Q4_K_M11.6 GB
GGUF
QW
Huihui Qwen3.6 27B Abliterated
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF
NO
nousresearch_nous-capybara-34b
Nous · TheBloke
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
DA
Darwin 36B Opus
Darwin · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
Reasoning
DE
DeepSeek R1 Distill Qwen 14B Abliterated
DeepSeek · mradermacher
Params14.8B
Context128K
Q4_K_M9.0 GB
Reasoning
NE
Nemotron 3 Nano 30B A3B
Nemotron · bartowski
Params31.6B
Context1024K
Q4_K_M24.7 GB
Coding
QW
Qwen3.6 40B Fable Fusion 6 Core Deckard Eleanor Heretic Uncensored NM DAU NEO MAX MTP
Qwen · DavidAU
Params39.1B
Context256K
Q4_K_M25.9 GB
CodingReasoningRoleplay
AY
Aya Expanse 8b
Aya · bartowski
Params8B
Context8K
Q4_K_M5.1 GB
GGUF
OL
Olmo 3.1 32B Instruct
OLMo · bartowski
Params32.2B
Context64K
Q4_K_M19.5 GB
GGUF
MI
MiniCPM V 4.7 35B A3B
MiniCPM · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
Vision
MI
Mistral 7B Instruct v0.3
Mistral · QuantFactory
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
PA
Parakeet TDT-CTC 110M
Parakeet · handy-computer
Params114M
Context—
Q4_K_M0.1 GB
GGUF
DE
Deepseek-Prover-V2-7B
DeepSeek · unsloth
Params6.91B
Context32K
Q4_K_M4.2 GB
GGUF
SP
Spark X2.6 4B LocalAI
Spark · weareapexcreators
Params4.45B
Context1024K
Q4_K_M3.5 GB
ReasoningTool use
GE
Gemma 4 E4B
Gemma · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
Vision
NA
VisionPsy Nano 460M GGUFs
Nanovlm · qvac
Params409M
Context8K
Q4_K_M0.3 GB
Vision
GE
Gemma 4 MoonGem 31B
Gemma · bartowski
Params30.7B
Context256K
Q4_K_M19.5 GB
RoleplayVision
PR
PreciseCoder 9B
Precisecoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Coding
LL
DaringFortitude
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
CY
Cyberdonia 24B v8
Cyberdonia · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
Roleplay
GE
gemma-2-9b-it
Gemma · lmstudio-community
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
HU
HunyuanVideo 1.5 T2V 480p
Hunyuanvideo · jayn7
Params8.33B
Context—
Q4_K_M5.1 GB
GGUF
QW
Qwen2.5 Coder 32B Instruct
Qwen · unsloth
Params32.8B
Context128K
Q4_K_M19.9 GB
Coding
WI
Wicked Stuff 26B A4B v2.0 Final
Wicked · ReadyArt
Params25.2B
Context256K
Q4_K_M16.8 GB
ReasoningRoleplayTool use
HI
HiDream I1 Full
Hidream · city96
Params17.1B
Context—
Q4_K_M11.5 GB
GGUF

Other quants

Keep browsing