Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3 Reranker 4B
Qwen · giladgd
Params4B
Context40K
Q4_K_M2.5 GB
DR
DrakIdol Roleplayer 1.0
Drakidol · mradermacher
Params3.88B
Context128K
Q4_K_M2.5 GB
AT
Athene codegemma 2 7b it alpaca v1.3 i1
Athene · mradermacher
Params8.54B
Context8K
Q4_K_M5.3 GB
GE
gemma 3 4b instruct
Gemma · lm-kit
Params3.88B
Context128K
Q4_K_M2.5 GB
LL
Llama Calix CLM 1.0
Llama · calix-ai-labs
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama 3.1 Tulu 3 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
MI
Writing_Partner_Mistral_7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 4B
Qwen · Aldaris
Params4B
Context40K
Q4_K_M2.5 GB
QW
Qwen3.5 9B Star Trek TNG DS9 Heretic Uncensored Thinking MAX NEOCODE
Qwen · DavidAU
Params8.95B
Context256K
Q4_K_M6.8 GB
MI
Open StarLake Swap 7B
Mistral · Aryanne
Params7.24B
Context32K
Smallest4.1 GB
MI
cognitive-hacker-gguf-0.2
Mistral · ebowwa
Params7.25B
Context32K
Q4_K_M4.4 GB
QW
Qwen3-Vl-4B-Thinking-1M
Qwen · unsloth
Params4B
Context977K
Q4_K_M2.5 GB
BI
Bielik 7B Instruct v0.1
Bielik · mradermacher
Params7.24B
Context4K
Q4_K_M4.4 GB
NE
Neural AlphaMistral 7B
Neural · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
YI
Yi 1.5 6B Chat
Yi · QuantFactory
Params6.06B
Context4K
Q4_K_M3.7 GB
QW
Qwen3.5 9B The Defiant Fable Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · luoyumo
Params8.95B
Context256K
Q4_K_M6.8 GB
ST
stabilityai japanese stablelm instruct gamma 7b
Stabilityai · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
LI
Lil Beastie 4b
Lil · mradermacher
Params4.02B
Context256K
Q4_K_M2.5 GB
NE
Neuron 4B Instruct
Neuron · mradermacher
Params4.02B
Context256K
Q4_K_M2.5 GB
TO
tokyotech llm Swallow 7b instruct v0.1
Tokyotech · mmnga
Params6.83B
Context4K
Q4_K_M4.1 GB
HU
Hunyuan 4B Instruct
Hunyuan · bartowski
Params4.22B
Context256K
Q4_K_M2.6 GB
LL
Colossal LLaMA 2 7b Base
Llama · mradermacher
Params7.04B
Context4K
Q4_K_M4.3 GB
QW
Qwen3 VL 8B Instruct
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
GR
Granite 4.1 8b Abliterated AND Disinhibited
Granite · mradermacher
Params8.38B
Context128K
Q4_K_M5.1 GB
GL
Sentie1.0 3B Claude Fable 5 GPT5.2 Sol Kimi K3 GLM 5.2
GLM · SentieAI
Params3.93B
Context256K
Q4_K_M2.4 GB
QW
QiMing Holos Plus Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
OL
Olmo-3-7B-Instruct
OLMo · Mungert
Params7.3B
Context64K
Q4_K_M4.7 GB
ME
steve-cse_mellogpt
Mellogpt · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LI
LightOnOCR 3 4B
Lightonocr · richardyoung
Params4.2B
Context256K
Q4_K_M2.7 GB
LL
Mini Lama 8B Base
Llama · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB