Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
WA
Wan2.2 Fun 5B Control
Wan · QuantStack
Params5B
Context—
Q4_K_M3.4 GB
GGUF
DE
DeepSeek R1 Distill Llama 8B Abliterated
DeepSeek · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Reasoning
PH
Phi 3 mini 4k instruct update2024 07 03 IMat
Phi · legraphista
Params3.82B
Context4K
Smallest0.8 GB
Coding
WI
Wingless_Imp_8B
Wingless · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen3-4B-2507-Instruct-Uncensored-HauhauCS-Aggressive
Qwen · HauhauCS
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
GE
Ablation 187 Gemma3.Webnovels.Test.Gemma 3 4b
Gemma · mradermacher
Params3.88B
Context128K
Q4_K_M2.5 GB
GGUF
QW
Qwen3.5 9B Heretic v2
Qwen · AIImageStudio
Params8.95B
Context256K
Smallest9.5 GB
Vision
LL
ggml llava v1.5 7b
LLaVA · mys
Params6.74B
Context4K
Smallest4.1 GB
Vision
PH
phi3.5 gutenberg 4B
Phi · bartowski
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LI
Ling 3.0 tiny abliterated APEX
Ling · SC117
Params7.9B
Context128K
Smallest3.4 GB
GGUF
QW
Huihui Qwen3 8B Abliterated v2
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
DA
L3 Dark Planet 8B
Dark · DavidAU
Params8B
Context8K
Q4_K_M5.1 GB
Roleplay
QW
Qwen2.5 VL 7B Abliterated Caption It
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
QW
Qwen3 VL Reranker 8B
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
EmbeddingVision
OL
ollama3.1
Ollama · xzlinuxmodels
Params8B
Context128K
Smallest6.6 GB
GGUF
OR
Ornith1.0 9B 3MPER0RR Abliterated
Ornith · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
ON
OneJev 4B
Onejev · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
Vision
PH
Phi 4 Mini Reasoning
Phi · lmstudio-community
Params3.84B
Context128K
Q4_K_M2.5 GB
CodingReasoning
ST
berkeley-nest_starling-lm-7b-alpha
Starling · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
JE
JEV 9B
Jev · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
QW
Qwen3.5 4B Heretic
Qwen · darrellbest
Params4.33B
Context256K
Q4_K_M2.8 GB
Vision
MY
Mythos v2 9b
Mythos · fableforge-ai
Params9.2B
Context256K
Q4_K_M5.8 GB
Tool use
HO
Holo 3.1 4B
Holo · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
ReasoningTool useVision
NE
Nemotron3-Nano-4B-Uncensored-HauhauCS-Aggressive
Nemotron · HauhauCS
Params3.97B
Context1024K
Q4_K_M2.8 GB
GGUF
TO
tokyotech llm Swallow MS 7b instruct v0.1
Tokyotech · mmnga
Params7.33B
Context4K
Q4_K_M4.4 GB
GGUF
QW
qwen2-7b-instruct
Qwen · Qwen
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen3.5 9B Cold Fusion GAIN v1.0 Uncensored Heretic NEO MAX Imatrix
Qwen · DavidAU
Params8.95B
Context256K
Q4_K_M6.8 GB
CodingReasoningRoleplay
QW
Qwen Image 2.1 Uncensored Genesis BF16
Qwen · LuffyTheFox
Params7.12B
Context—
Smallest14.2 GB
GGUF
GE
Gemma 4 E2B It Heretic Ara
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
GGUF
QW
Qwen2 VL 7B Instruct
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
Vision

Other sizes

Keep browsing