Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5 4B Anon Reasoning v2
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
SU
SummLlama3.1 8B
Summllama · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
GE
Gemma 2 Ataraxy 9B
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
FI
Fin R1
Fin · mradermacher
Params7.6B
Context32K
Q4_K_M4.7 GB
Reasoning
CH
Chocolatine 3B Instruct DPO v1.2
Chocolatine · bartowski
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llm Jp 3 3.7b
Llm · mmnga
Params3.78B
Context4K
Q4_K_M2.3 GB
GGUF
QW
Huihui Qwen3.5 4B Claude 4.6 Opus Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningVision
QW
imajev 4b
Qwen · mindchain
Params4.2B
Context256K
Q4_K_M2.7 GB
Vision
HU
Huihui OmniCoder 9B Abliterated
Huihui · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Coding
QW
SummOrchestra Qwen3 8B GRPO BRLP SAMSUM
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
Roleplay
QW
Qwen3.5 9B Claude Fable 5xMerge
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoning
GE
Gemma 4 E4B It Erotica
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
GGUF
KU
Kunokukulemonchini 7b
Kunokukulemonchini · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
TO
TooManyMixRolePlay-7B-Story_V3.5
Toomanymixroleplay · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
QW
Qwen3.5 9B Uncensored
Qwen · ccharnkij
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
QW
Qwen Image 2.1 Uncensored
Qwen · Janchan123
Params7.12B
Context—
Q4_K_M4.6 GB
GGUF
MI
nousresearch_yarn-mistral-7b-128k
Mistral · TheBloke
Params7.24B
Context128K
Q4_K_M4.4 GB
GGUF
WE
WeVisOne VL 4B
Wevisone · mradermacher
Params4.42B
Context256K
Q4_K_M2.7 GB
Tool useVision
NO
Noema 1.5 4B Fast
Noema · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
CodingReasoning
MI
TowerInstruct Mistral 7B v0.2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TO
ToriiGate 0.5
Toriigate · DraconicDragon
Params4.84B
Context256K
Q4_K_M3.1 GB
Vision
GL
glm-4-9b-chat
GLM · legraphista
Params9.4B
Context128K
Smallest3.1 GB
GGUF
QW
Qwen Image 2.1 Text Encoder Heretic
Qwen · chfm
Params8.2B
Context256K
Q4_K_M5.0 GB
CodingVision
DA
danube3-4B
Danube · h2oai
Params3.96B
Context8K
Q4_K_M2.4 GB
GGUF
CH
UnCanny Photorealism Chroma
Chroma · dawncreates
Params8.9B
Context—
Q4_K_M5.6 GB
GGUF
LL
Humanish Roleplay Llama 3.1 8B i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
NE
Neuralgamma 2.5 7B slerp
Neuralgamma · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
Tool use
QW
Qwen3.8 4B Agentic Distill
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
FR
FrogNano 4B 2609
Frognano · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
GGUF

Other sizes

Keep browsing