Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen Image 2.1 Text Encoder Heretic DHQ
Qwen · pottokao
Params8.2B
Context256K
Q4_K_M5.0 GB
CodingVision
QW
Qwen Image 2.1 viggle turbo v0.3 6step
Qwen · Abiray
Params7.12B
Context—
Q4_K_M4.2 GB
GGUF
ST
L3 8B Stheno v3.2
Stheno · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
JA
Jan nano
Jan · Menlo
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
HO
K2-Horizon-3.7B
Horizon · mradermacher
Params5.06B
Context512K
Q4_K_M3.2 GB
GGUF
QW
Qwen3-4B-abliterated
Qwen · Mungert
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
LL
Llama 7B rollercoaster v2 i1
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
nondzu_mistral-7b-instruct-v0.2-code-ft
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
FI
SUFE AIFLM Lab Fin R1
Fin · bartowski
Params7.6B
Context32K
Q4_K_M4.7 GB
Reasoning
PH
phi-4-mini-instruct
Phi · Mungert
Params3.84B
Context128K
Q4_K_M2.5 GB
CodingReasoning
QW
Qwen3.5 9B The Bradbury F451 Pro Writer Uncensored Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingRoleplayVision
QW
Qwen3.5 9B C3SM SDM Agentic Coder
Qwen · OliviaRossi
Params8.95B
Context256K
Smallest6.5 GB
CodingTool use
QW
Qwen3 VL 8B Instruct Abliterated v2
Qwen · qign
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
HO
Holo 3.1 4B
Holo · prithivMLmods
Params4.84B
Context256K
Q4_K_M3.1 GB
Vision
QW
Qwen3 VL 8B Instruct Heretic
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
SE
SenseNova U1 8B MoT SFT
Sensenova · mradermacher
Params9.35B
Context256K
Q4_K_M5.7 GB
CodingReasoningTool use
GR
Kiesel v2
Granite · mradermacher
Params8.8B
Context128K
Q4_K_M5.3 GB
ReasoningTool use
LL
bangla-llama-7b-instruct-v0.1
Llama · mradermacher
Params6.89B
Context4K
Q4_K_M4.2 GB
GGUF
HO
K2 Horizon 3.7B
Horizon · NANI-Nithin
Params5.06B
Context512K
Q4_K_M3.2 GB
GGUF
QW
Eve V2 Unleashed Qwen3.5 8B Liberated 4K 4B Merged
Qwen · JeffGreen311
Params4.66B
Context256K
Q4_K_M3.4 GB
CodingTool use
QW
Qwen3.5 4B
Qwen · gguf
Params4.33B
Context256K
Q4_K_M2.8 GB
Vision
GE
Huihui Gemma 3n E4B It Abliterated
Gemma · bartowski
Params6.9B
Context32K
Q4_K_M4.2 GB
GGUF
QW
Qwen3.5 9B Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
MO
MolOptAgent 7B
Moloptagent · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
DE
DeepSeek R1 Qwen3 8B heretic
DeepSeek · saidutta69
Params8.19B
Context128K
Q4_K_M5.0 GB
ReasoningRoleplayTool use
LL
llava v1.5 7b llamafile
Llama · mozilla-ai
Params6.74B
Context4K
Smallest4.1 GB
Vision
GE
Gemma 4 E2B
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
GGUF
MI
Mimo-Vl-7B-Rl
Mimo · unsloth
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
DE
Qwen3.5 9B Distill Deepseek V4.1 Flash 20261003
DeepSeek · kevinzhou21
Params9.2B
Context256K
Q4_K_M5.8 GB
Vision
QW
Qwen3.8 9B Distill SLERP F451 Pro Writer Uncensored
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
RoleplayVision

Other sizes

Keep browsing