Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5 9B Heretic Ara v1
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
QW
Qwen3 VL 8B Instruct Unredacted MAX
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
GE
Gemma 3 4b It Uncensored DBL X
Gemma · mradermacher
Params4.26B
Context128K
Q4_K_M2.7 GB
Roleplay
SY
migtissera_synthia-7b-v2.0
Synthia · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 9B Heretic v2
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
MI
openbuddy_openbuddy-mistral-7b-v13.1
Mistral · TheBloke
Params7.28B
Context32K
Q4_K_M4.4 GB
GGUF
SP
Spark X2.6 4B Heretic
Spark · mradermacher
Params4.11B
Context1024K
Q4_K_M2.6 GB
ReasoningTool use
MI
Young Children Storyteller Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
KA
KangalKhan RawRuby 7B i1
Kangalkhan · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MO
MonarchPipe 7B slerp
Monarchpipe · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
LL
Llama 3 8B ProLong 64k Base
Llama · RichardErkhov
Params8B
Context64K
Q4_K_M4.9 GB
GGUF
FI
Fino1 8B
Fino · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
ReasoningTool use
YI
Yi-6B-200K
Yi · mradermacher
Params6.06B
Context195K
Q4_K_M3.7 GB
GGUF
FR
fredithefish_openzephyrchat-v0.2
Fredithefish · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LE
sanjiwatsuki_lelantos-7b
Lelantos · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OL
Olmo 3 7B Instruct DPO
OLMo · mradermacher
Params7.3B
Context64K
Q4_K_M4.5 GB
Reasoning
IN
Internlm2_5 7b Chat 1m
InternLM · duyntnet
Params7.74B
Context256K
Q4_K_M4.7 GB
GGUF
IN
Index Translate 9B
Index · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
GGUF
LL
Llama-3-Patronus-Lynx-8B-Instruct
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Flirty Mistral
Mistral · mradermacher
Params7.3B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 4B Instruct 2507 Heretic v4
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
TR
Triangulum 5B It
Triangulum · mradermacher
Params5.41B
Context8K
Q4_K_M3.4 GB
Reasoning
DE
DeepSeek R1 Distill Qwen 7B Abliterated Obliteratus
DeepSeek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
Reasoning
QW
ReSearch Qwen 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
AT
Athena-CodePhi-7B-4k-v0.1
Athena · mradermacher
Params3.82B
Context4K
Q4_K_M2.3 GB
Coding
IM
Z images models
Images · Andyhere
Params4.02B
Context40K
Smallest5.1 GB
ReasoningTool use
KU
Kuchiki 1.1 L2 7B
Kuchiki · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
NE
NeuralGanesha 7b
Neuralganesha · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
SP
Spark X2.6 4B Tema_Q Abliterated
Spark · mradermacher
Params4.11B
Context1024K
Q4_K_M2.6 GB
ReasoningTool use
LL
Llama 3.1 Storm 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
Reasoning

Other sizes

Keep browsing