Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
OR
Ornith-1.5-9B
Ornith · ornith-ai
Params9.2B
Context256K
Q4_K_M5.8 GB
Vision
LO
locate anything.cpp
Locateanything · mudler
Params3.83B
Context—
Smallest4.7 GB
GGUF
OR
Ornith 1.0 9B
Ornith · ornith-ai
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
QW
Qwen Image 2.1 Uncensored Quantized
Qwen · abenzerps
Params7.12B
Context—
Q4_K_M4.6 GB
GGUF
QW
Qwen3.5 9B The Defiant Fable Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · DavidAU
Params8.95B
Context256K
Q4_K_M6.8 GB
CodingReasoningRoleplay
MO
MOSS Music 8B Instruct
Moss · scragnog
Params8.2B
Context—
Smallest1.7 GB
GGUF
OR
Ornith 1.5 9B Uncensored
Ornith · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
QW
Qwen Image 2.1
Qwen · unsloth
Params7.12B
Context—
Q4_K_M4.2 GB
GGUF
QW
Qwen3.5-9B
Qwen · unsloth
Params8.95B
Context256K
Q4_K_M5.7 GB
Vision
QW
Qwen3.5-4B
Qwen · unsloth
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
QW
Qwen3Vl 8b Instruct
Qwen · Qwen
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
WA
Wan2.1
Wan · DeepBeepMeep
Params9.2B
Context256K
Q4_K_M0.3 GB
GGUF
IM
Z Image Turbo
Image · unsloth
Params6.15B
Context—
Q4_K_M3.9 GB
GGUF
GE
gemma 4 E4B it qat q4 0
Gemma · google
Params7.46B
Context128K
Smallest5.2 GB
Vision
HY
Hy MT2 7B
tencent
Params7.5B
Context256K
Q4_K_M4.6 GB
GGUF
QW
MiMo V2.6 Distill Qwen 9B
Qwen · bartowski
Params8.95B
Context256K
Q4_K_M5.8 GB
CodingTool useVision
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · HauhauCS
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
CY
Cyber Ornith 1.5 9B OBLITERATED
Cyber · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningTool use
SP
Sharp Spark X2.5 4B
Spark · peculiar-ragdoll
Params4.11B
Context—
Smallest0.0 GB
Reasoning
HI
Higgs Audio v3 Studio
Higgs · drbaph
Params4.65B
Context—
Q4_K_M4.1 GB
GGUF
QW
Qwen Image 2.1 Turbo
Qwen · unsloth
Params7.12B
Context—
Q4_K_M4.2 GB
GGUF
QW
Qwen3 8B Awq Compatible Instruct
Qwen · Qwen
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
QW
Qwen3 VL 8B Instruct Abliterated
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
QW
Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16
Qwen · HauhauCS
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
QW
qwen2.5-7b-instruct
Qwen · Qwen
Params7.6B
Context128K
Q4_K_M4.7 GB
GGUF
QW
Qwen3 4B Instruct Awq
Qwen · Qwen
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
MI
MiniMax-Music
Minimax · audio-cpp
Params4.9B
Context—
Smallest0.1 GB
GGUF
QW
Qwen Image 2.1 Turbo Uncensored
Qwen · AtomicChat
Params8.19B
Context256K
Smallest3.3 GB
CodingTool useVision
QW
Qwen3.8 4B
Qwen · empero-ai
Params4.33B
Context256K
Q4_K_M2.8 GB
Reasoning
QW
Qwen3Vl 4b Instruct
Qwen · Qwen
Params4B
Context256K
Q4_K_M2.5 GB
Vision

What counts as a small model?

On this site small means roughly 4B to 9B parameters. A Q4_K_M file is usually 2 to 6 GB, which fits an 8 GB GPU with room for context. 91,171 models fit this class. The most downloaded are Ornith-1.5-9B, locate anything.cpp and Ornith 1.0 9B.

Their Q4_K_M files run 0.0 to 42.5 GB. The families you will see most are Llama (17914), Qwen (13245) and Mistral (6032). The size class sorts models by parameter count, so a model with more experts or a longer context can need more memory than its class suggests. The model pages show the real file sizes.

Frequently asked questions

How much VRAM does a small GGUF model need?

A Q4_K_M file is usually 2 to 6 GB, which fits an 8 GB GPU with room for context.

Which small GGUF model is the most popular?

By Hugging Face downloads, Ornith-1.5-9B.

Other sizes

Keep browsing