Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Qwen Image 2.1 Text Encoder Heretic DHQ
Qwen · pottokao
Params8.2B
Context256K
Q4_K_M5.0 GB
QW
Qwen Image 2.1 viggle turbo v0.3 6step
Qwen · Abiray
Params7.12B
Context—
Q4_K_M4.2 GB
ST
L3 8B Stheno v3.2
Stheno · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
JA
Jan nano
Jan · Menlo
Params4B
Context40K
Q4_K_M2.5 GB
HO
K2-Horizon-3.7B
Horizon · mradermacher
Params5.06B
Context512K
Q4_K_M3.2 GB
QW
Qwen3-4B-abliterated
Qwen · Mungert
Params4B
Context40K
Q4_K_M2.5 GB
LL
Llama 7B rollercoaster v2 i1
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
nondzu_mistral-7b-instruct-v0.2-code-ft
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
FI
SUFE AIFLM Lab Fin R1
Fin · bartowski
Params7.6B
Context32K
Q4_K_M4.7 GB
PH
phi-4-mini-instruct
Phi · Mungert
Params3.84B
Context128K
Q4_K_M2.5 GB
QW
Qwen3.5 9B The Bradbury F451 Pro Writer Uncensored Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen3.5 9B C3SM SDM Agentic Coder
Qwen · OliviaRossi
Params8.95B
Context256K
Smallest6.5 GB
QW
Qwen3 VL 8B Instruct Abliterated v2
Qwen · qign
Params8.19B
Context256K
Q4_K_M5.0 GB
HO
Holo 3.1 4B
Holo · prithivMLmods
Params4.84B
Context256K
Q4_K_M3.1 GB
QW
Qwen3 VL 8B Instruct Heretic
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
SE
SenseNova U1 8B MoT SFT
Sensenova · mradermacher
Params9.35B
Context256K
Q4_K_M5.7 GB
GR
Kiesel v2
Granite · mradermacher
Params8.8B
Context128K
Q4_K_M5.3 GB
LL
bangla-llama-7b-instruct-v0.1
Llama · mradermacher
Params6.89B
Context4K
Q4_K_M4.2 GB
HO
K2 Horizon 3.7B
Horizon · NANI-Nithin
Params5.06B
Context512K
Q4_K_M3.2 GB
QW
Eve V2 Unleashed Qwen3.5 8B Liberated 4K 4B Merged
Qwen · JeffGreen311
Params4.66B
Context256K
Q4_K_M3.4 GB
QW
Qwen3.5 4B
Qwen · gguf
Params4.33B
Context256K
Q4_K_M2.8 GB
GE
Huihui Gemma 3n E4B It Abliterated
Gemma · bartowski
Params6.9B
Context32K
Q4_K_M4.2 GB
QW
Qwen3.5 9B Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
MO
MolOptAgent 7B
Moloptagent · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
DE
DeepSeek R1 Qwen3 8B heretic
DeepSeek · saidutta69
Params8.19B
Context128K
Q4_K_M5.0 GB
LL
llava v1.5 7b llamafile
Llama · mozilla-ai
Params6.74B
Context4K
Smallest4.1 GB
GE
Gemma 4 E2B
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
MI
Mimo-Vl-7B-Rl
Mimo · unsloth
Params7.62B
Context125K
Q4_K_M4.7 GB
DE
Qwen3.5 9B Distill Deepseek V4.1 Flash 20261003
DeepSeek · kevinzhou21
Params9.2B
Context256K
Q4_K_M5.8 GB
QW
Qwen3.8 9B Distill SLERP F451 Pro Writer Uncensored
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB