Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
TE
Ternary Bonsai 2 27B Speculative Drafters
Ternary · NOhhh
Params1.92B
Context2K
Smallest0.0 GB
GGUF
LI
lightblue reranker 0.5 cont filt
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
EmbeddingTool use
RF
rfdetr cpp seg 2xlarge
Rfdetr · mudler
Params38M
Context—
Smallest0.0 GB
GGUF
UN
Unbabel TowerInstruct 7B v0.1
Unbabel · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama3 Med42 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.5 2B
Qwen · titan0115
Params1.88B
Context256K
Q4_K_M1.3 GB
ReasoningTool use
LL
chinese llama 2 7b 64k
Llama · hfl
Params6.93B
Context64K
Smallest2.6 GB
GGUF
LL
YBXL Med LLaMA3 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · RusteddDDD
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
ST
stt de fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
GGUF
LL
meta llama Llama 2 7b hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
JI
jieliu Storm 7B
Jieliu · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
GI
Gille StrangeMerges 21 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FI
Finance Chat
Finance · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
CodeLlama-70b-Python-hf
Llama · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
Coding
ML
mlfoundations dev oh teknium scaling down ratiocontrolled 1.0
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 3 Nephilim v2.1 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GO
google shieldgemma 9b
Google · RichardErkhov
Params9.24B
Context8K
Q4_K_M3.1 GB
GGUF
QU
quds-v4-fa-f16
Quds · cstr
Params114M
Context—
Smallest0.1 GB
GGUF
QW
Qwen3.5 27B Opus Reasoning v2 Abliterated EvoPress
Qwen · Flakily6416
Params26.9B
Context256K
Smallest10.7 GB
ReasoningTool useVision
LL
Llama 4 Scout 17B 16E Instruct
Llama · steampunque
Params108B
Context10240K
Smallest42.8 GB
Tool use
GR
Granite 3b Code Instruct 2k
Granite · RichardErkhov
Params3.48B
Context2K
Q4_K_M2.1 GB
Coding
RF
rfdetr cpp seg large
Rfdetr · mudler
Params35M
Context—
Smallest0.0 GB
GGUF
MI
macadeliccc Mistral 7B v0.2 OpenHermes
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
GE
Gemma 2 Ataraxy Gemmasutra 9B slerp
Gemma · AI-Art-Collab
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
FI
finance chat
Finance · andrijdavid
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MA
Magic-Dolphin-7b
Magic · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
stt ar fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
GGUF
LL
llama 3 8b English to Hinglish
Llama · Hinglish-Project
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing