Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3 Reranker 4B
Qwen · giladgd
Params4B
Context40K
Q4_K_M2.5 GB
Embedding
DR
DrakIdol Roleplayer 1.0
Drakidol · mradermacher
Params3.88B
Context128K
Q4_K_M2.5 GB
Roleplay
AT
Athene codegemma 2 7b it alpaca v1.3 i1
Athene · mradermacher
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
GE
gemma 3 4b instruct
Gemma · lm-kit
Params3.88B
Context128K
Q4_K_M2.5 GB
GGUF
LL
Llama Calix CLM 1.0
Llama · calix-ai-labs
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama 3.1 Tulu 3 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Writing_Partner_Mistral_7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 4B
Qwen · Aldaris
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
QW
Qwen3.5 9B Star Trek TNG DS9 Heretic Uncensored Thinking MAX NEOCODE
Qwen · DavidAU
Params8.95B
Context256K
Q4_K_M6.8 GB
ReasoningRoleplayVision
MI
Open StarLake Swap 7B
Mistral · Aryanne
Params7.24B
Context32K
Smallest4.1 GB
Tool use
MI
cognitive-hacker-gguf-0.2
Mistral · ebowwa
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3-Vl-4B-Thinking-1M
Qwen · unsloth
Params4B
Context977K
Q4_K_M2.5 GB
ReasoningVision
BI
Bielik 7B Instruct v0.1
Bielik · mradermacher
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
NE
Neural AlphaMistral 7B
Neural · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
YI
Yi 1.5 6B Chat
Yi · QuantFactory
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
QW
Qwen3.5 9B The Defiant Fable Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · luoyumo
Params8.95B
Context256K
Q4_K_M6.8 GB
CodingReasoningRoleplay
ST
stabilityai japanese stablelm instruct gamma 7b
Stabilityai · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
LI
Lil Beastie 4b
Lil · mradermacher
Params4.02B
Context256K
Q4_K_M2.5 GB
ReasoningTool use
NE
Neuron 4B Instruct
Neuron · mradermacher
Params4.02B
Context256K
Q4_K_M2.5 GB
Tool use
TO
tokyotech llm Swallow 7b instruct v0.1
Tokyotech · mmnga
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
HU
Hunyuan 4B Instruct
Hunyuan · bartowski
Params4.22B
Context256K
Q4_K_M2.6 GB
GGUF
LL
Colossal LLaMA 2 7b Base
Llama · mradermacher
Params7.04B
Context4K
Q4_K_M4.3 GB
GGUF
QW
Qwen3 VL 8B Instruct
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
GR
Granite 4.1 8b Abliterated AND Disinhibited
Granite · mradermacher
Params8.38B
Context128K
Q4_K_M5.1 GB
GGUF
GL
Sentie1.0 3B Claude Fable 5 GPT5.2 Sol Kimi K3 GLM 5.2
GLM · SentieAI
Params3.93B
Context256K
Q4_K_M2.4 GB
CodingReasoningTool use
QW
QiMing Holos Plus Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
OL
Olmo-3-7B-Instruct
OLMo · Mungert
Params7.3B
Context64K
Q4_K_M4.7 GB
Tool use
ME
steve-cse_mellogpt
Mellogpt · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LI
LightOnOCR 3 4B
Lightonocr · richardyoung
Params4.2B
Context256K
Q4_K_M2.7 GB
Vision
LL
Mini Lama 8B Base
Llama · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
Vision

Other sizes

Keep browsing