Q4_K_M GGUF models

4-bit k-quant, medium. The common default: a good balance of size and quality. 148631 models have a Q4_K_M file. Sizes run from 1M to 2450B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
FL
FLUX.1 dev
Flux · unsloth
Params11.9B
Context—
Q4_K_M6.9 GB
GGUF
QW
Qwen2.5 72B Instruct Abliterated
Qwen · zetasepic
Params72.7B
Context32K
Q4_K_M47.4 GB
GGUF
CO
Models 01 Ai Yi Coder 1.5B Chat
Coder · MaziyarPanahi
Params1.48B
Context128K
Q4_K_M1.0 GB
Coding
QW
Qwen2.5 VL 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
QW
Huihui Qwen3 30B A3B Instruct 2507 Abliterated
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
GGUF
SE
SeedVR2
Seedvr · cmeka
Params3.39B
Context—
Q4_K_M2.0 GB
GGUF
MI
Models Mistralai Ministral 3 3B Reasoning 2512
Mistral · MaziyarPanahi
Params3.43B
Context256K
Q4_K_M2.1 GB
Reasoning
GE
Models Google Gemma 2 2b It
Gemma · MaziyarPanahi
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
CL
Clef Flash
Clef · bartowski
Params9.1B
Context256K
Q4_K_M6.0 GB
CodingVision
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
MI
MiniCPM V 4 5
MiniCPM · openbmb
Params8.2B
Context40K
Q4_K_M5.0 GB
CodingVision
GE
Gemma 4 E4B It OBLITERATED v3.1
Gemma · OBLITERATUS
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
WI
models--microsoft--WizardLM-2-7B
Wizardlm · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Models Google Gemma 3 1b It
Gemma · MaziyarPanahi
Params1B
Context32K
Q4_K_M0.8 GB
GGUF
CL
Cloudflare clef
Clef · bartowski
Params27B
Context256K
Q4_K_M17.7 GB
CodingVision
BE
BeaverAI NeMoist 21B v1a
Beaverai · RichardErkhov
Params21B
Context—
Q4_K_M0.0 GB
GGUF
QW
Qwen Image 2.1 Uncensored
Qwen · 0xSojalSec
Params7.12B
Context—
Q4_K_M4.6 GB
GGUF
GR
Granite Speech 4.1 2B
Granite · handy-computer
Params2.31B
Context—
Q4_K_M1.6 GB
GGUF
QW
Qwen3.8 27B Heretic JP Roleplay NSFW DanbooruTags
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
Roleplay
MI
mistralai_Devstral Small 2 24B Instruct 2512
Mistral · lmstudio-community
Params23.6B
Context384K
Q4_K_M14.3 GB
CodingVision
MI
Models Mrfakename Mistral Small 3.1 24b Instruct 2503 Hf
Mistral · MaziyarPanahi
Params23.6B
Context32K
Q4_K_M14.3 GB
GGUF
FU
Fun ASR MLT Nano 2512
Fun · handy-computer
Params830M
Context—
Q4_K_M0.6 GB
GGUF
QW
Qwen3 TTS 12Hz 1.7B Base
Qwen · ggml-org
Params1.73B
Context32K
Q4_K_M1.0 GB
Vision
LT
LTX 2
Ltx · unsloth
Params18.9B
Context—
Q4_K_M12.8 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · NousResearch
Params8B
Context128K
Q4_K_M4.9 GB
RoleplayTool use
DE
DeepSeek R1 Distill Qwen 14B
DeepSeek · unsloth
Params14.8B
Context128K
Q4_K_M9.0 GB
Reasoning
YI
models--01-ai--Yi-1.5-6B-Chat
Yi · MaziyarPanahi
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
GE
Models Google Gemma 3 12b It
Gemma · MaziyarPanahi
Params11.8B
Context128K
Q4_K_M7.3 GB
GGUF
LL
models--MaziyarPanahi--Llama-3-8B-Instruct-64k
Llama · MaziyarPanahi
Params8B
Context63K
Q4_K_M4.9 GB
GGUF
MI
Models Mistralai Mistral Nemo Instruct 2407
Mistral · MaziyarPanahi
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF

Other quants

Keep browsing