Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen_Qwen3.5 9B
Qwen · lmstudio-community
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
YU
YuE2 LM
Yue · scragnog
Params3.8B
Context24K
Q4_K_M2.4 GB
GGUF
GE
Gemma-4-E4B-It
Gemma · unsloth
Params7.5B
Context128K
Q4_K_M5.0 GB
Vision
QW
Qwen3.8 9B
Qwen · empero-ai
Params9.2B
Context256K
Q4_K_M5.8 GB
Reasoning
QW
Qwen3-4B
Qwen · unsloth
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
UM
Umt5 Xxl
Umt · city96
Params5.68B
Context512
Q4_K_M3.7 GB
Coding
GE
Gemma-4-E2B-It
Gemma · unsloth
Params4.65B
Context128K
Q4_K_M3.1 GB
Vision
QW
Ternary Bonsai 27B
Qwen · prism-ml
Params3.6B
Context4K
Smallest1.9 GB
Vision
GE
Gemma 4 E4B
Gemma · lmstudio-community
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
MI
Voxtral Mini 4B Realtime
Mistral · handy-computer
Params4.43B
Context—
Q4_K_M2.8 GB
GGUF
QW
Qwen3-4B-Instruct-2507
Qwen · unsloth
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
WA
Wan2.2 TI2V 5B
Wan · QuantStack
Params5B
Context—
Q4_K_M3.4 GB
GGUF
QW
Qwen3.5 9B Kimi K3 Distilled
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningVision
FL
FLUX.2 klein 4B
Flux · unsloth
Params3.88B
Context—
Q4_K_M2.6 GB
GGUF
QW
Qwen3-Vl-4B-Instruct
Qwen · unsloth
Params4B
Context256K
Q4_K_M2.5 GB
Vision
QW
Qwen Image 2.1 viggle turbo
Qwen · Viggle
Params7.12B
Context—
Q4_K_M4.3 GB
GGUF
B1
B1 9B
mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool useVision
FL
FLUX.2 klein 9B
Flux · unsloth
Params9.08B
Context—
Q4_K_M5.7 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
OR
Ornith-1.0-9B
Ornith · unsloth
Params8.95B
Context256K
Q4_K_M5.7 GB
Vision
WA
Wan2.2 TI2V 5B
Wan · unsloth
Params5B
Context—
Q4_K_M3.4 GB
GGUF
QW
Qwen Image 2.1 Turbo
Qwen · DogukanUrker
Params7.1B
Context—
Smallest2.4 GB
GGUF
QW
Qwythos-9B-v2
Qwythos · empero-ai
Params8.95B
Context1024K
Q4_K_M5.7 GB
ReasoningTool useVision
SP
Spark X2.5 4B
Spark · XHToken
Params4.1B
Context1024K
Q4_K_M2.6 GB
GGUF
SQ
sqlcoder 7b 2
Sqlcoder · defog
Params6.74B
Context16K
Smallest4.8 GB
Coding
QW
Qwythos 9B Claude Mythos 5 1M
Qwythos · empero-ai
Params8.95B
Context1024K
Q4_K_M5.6 GB
ReasoningTool useVision
SP
Spark X2.5 4B
Spark · abenzerps
Params4.1B
Context1024K
Q4_K_M2.6 GB
GGUF
OR
Ornith 1.5 9B OBLITERATED
Ornith · OBLITERATUS
Params9.2B
Context256K
Q4_K_M5.8 GB
Vision
TE
Ternary Bonsai 8B
Ternary · prism-ml
Params8.19B
Context64K
Smallest2.2 GB
GGUF
JA
Jan-v3.5-4B
Jan · janhq
Params4.4B
Context256K
Q4_K_M2.7 GB
GGUF

Other sizes

Keep browsing