Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
OpenChat 3.5 7B Qwen v2.0
Qwen · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MI
OpenHermes-2.5-Mistral-7B-x2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MI
Mahou-1.2-mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MA
Magnum Blackout Ataraxy 2 9b i1
Magnum · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LO
Loki v5.1 i1
Loki · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.8 4B Heretic
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
MI
Ministral 8B Instruct 2410
Mistral · bartowski
Params8B
Context32K
Q4_K_M4.9 GB
GGUF
JI
JiRackUltra_7b
Jirackultra · mradermacher
Params7.6B
Context128K
Q4_K_M4.7 GB
Tool use
LL
Llama 3.1 8B Reason Blend 888k i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
ZE
Zen3 Vl
Zen · mradermacher
Params8.2B
Context256K
Q4_K_M5.0 GB
Vision
OP
dreamgen_opus-v0-7b
Opus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · roleplaiapp
Params7.62B
Context128K
Q4_K_M4.7 GB
CodingReasoningRoleplay
QW
Qwen3.5 9B Brainwaves OneJev1.6
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
LL
Llama-3-Groq-8B-Tool-Use
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
Tool use
LL
Lexi Llama 3 8B Uncensored
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GR
Granite 4.1 8b
Granite · bartowski
Params8.79B
Context128K
Q4_K_M5.5 GB
GGUF
LL
Llama 3.1 Minitron 4B Width Base
Llama · ThomasBaruzier
Params4.51B
Context128K
Q4_K_M2.8 GB
GGUF
JE
JEV 9B
Jev · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
OP
OpenCerebrum 2.0 7B
Opencerebrum · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
LL
Asclepius-Llama3-8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.5-4B
Qwen · LocalScribe
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
NU
NuMarkdown-8B-Thinking
Numarkdown · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
ReasoningVision
GE
Gemma 4 E2B It Abliterated
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
GGUF
QW
Qwen3 8B Base
Qwen · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
GGUF
NE
NeuralMaths Experiment 7b
Neuralmaths · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
PL
plumb 4b
Plumb · crh225
Params4.2B
Context256K
Q4_K_M2.7 GB
GGUF
NE
Next 8b
Next · mradermacher
Params8.19B
Context128K
Q4_K_M5.0 GB
Reasoning
IN
Index Translate 9B
Index · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
Vision
LL
Cogito v1 Preview Llama 3B
Llama · bartowski
Params3.61B
Context128K
Q4_K_M2.2 GB
GGUF
GR
Grok Oss Thanatos
Grok · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF

Other sizes

Keep browsing