Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
WA
Wan2.2 Fun 5B Control
Wan · QuantStack
Params5B
Context—
Q4_K_M3.4 GB
DE
DeepSeek R1 Distill Llama 8B Abliterated
DeepSeek · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
PH
Phi 3 mini 4k instruct update2024 07 03 IMat
Phi · legraphista
Params3.82B
Context4K
Smallest0.8 GB
WI
Wingless_Imp_8B
Wingless · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen3-4B-2507-Instruct-Uncensored-HauhauCS-Aggressive
Qwen · HauhauCS
Params4B
Context256K
Q4_K_M2.5 GB
GE
Ablation 187 Gemma3.Webnovels.Test.Gemma 3 4b
Gemma · mradermacher
Params3.88B
Context128K
Q4_K_M2.5 GB
QW
Qwen3.5 9B Heretic v2
Qwen · AIImageStudio
Params8.95B
Context256K
Smallest9.5 GB
LL
ggml llava v1.5 7b
LLaVA · mys
Params6.74B
Context4K
Smallest4.1 GB
PH
phi3.5 gutenberg 4B
Phi · bartowski
Params3.82B
Context128K
Q4_K_M2.4 GB
LI
Ling 3.0 tiny abliterated APEX
Ling · SC117
Params7.9B
Context128K
Smallest3.4 GB
QW
Huihui Qwen3 8B Abliterated v2
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
DA
L3 Dark Planet 8B
Dark · DavidAU
Params8B
Context8K
Q4_K_M5.1 GB
QW
Qwen2.5 VL 7B Abliterated Caption It
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
QW
Qwen3 VL Reranker 8B
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
OL
ollama3.1
Ollama · xzlinuxmodels
Params8B
Context128K
Smallest6.6 GB
OR
Ornith1.0 9B 3MPER0RR Abliterated
Ornith · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ON
OneJev 4B
Onejev · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
PH
Phi 4 Mini Reasoning
Phi · lmstudio-community
Params3.84B
Context128K
Q4_K_M2.5 GB
ST
berkeley-nest_starling-lm-7b-alpha
Starling · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
JE
JEV 9B
Jev · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen3.5 4B Heretic
Qwen · darrellbest
Params4.33B
Context256K
Q4_K_M2.8 GB
MY
Mythos v2 9b
Mythos · fableforge-ai
Params9.2B
Context256K
Q4_K_M5.8 GB
HO
Holo 3.1 4B
Holo · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
NE
Nemotron3-Nano-4B-Uncensored-HauhauCS-Aggressive
Nemotron · HauhauCS
Params3.97B
Context1024K
Q4_K_M2.8 GB
TO
tokyotech llm Swallow MS 7b instruct v0.1
Tokyotech · mmnga
Params7.33B
Context4K
Q4_K_M4.4 GB
QW
qwen2-7b-instruct
Qwen · Qwen
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Qwen3.5 9B Cold Fusion GAIN v1.0 Uncensored Heretic NEO MAX Imatrix
Qwen · DavidAU
Params8.95B
Context256K
Q4_K_M6.8 GB
QW
Qwen Image 2.1 Uncensored Genesis BF16
Qwen · LuffyTheFox
Params7.12B
Context—
Smallest14.2 GB
GE
Gemma 4 E2B It Heretic Ara
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
QW
Qwen2 VL 7B Instruct
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB