Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3.5 4B Anon Reasoning v2
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
SU
SummLlama3.1 8B
Summllama · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
Gemma 2 Ataraxy 9B
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
FI
Fin R1
Fin · mradermacher
Params7.6B
Context32K
Q4_K_M4.7 GB
CH
Chocolatine 3B Instruct DPO v1.2
Chocolatine · bartowski
Params3.82B
Context128K
Q4_K_M2.4 GB
LL
Llm Jp 3 3.7b
Llm · mmnga
Params3.78B
Context4K
Q4_K_M2.3 GB
QW
Huihui Qwen3.5 4B Claude 4.6 Opus Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
QW
imajev 4b
Qwen · mindchain
Params4.2B
Context256K
Q4_K_M2.7 GB
HU
Huihui OmniCoder 9B Abliterated
Huihui · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
SummOrchestra Qwen3 8B GRPO BRLP SAMSUM
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
QW
Qwen3.5 9B Claude Fable 5xMerge
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GE
Gemma 4 E4B It Erotica
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
KU
Kunokukulemonchini 7b
Kunokukulemonchini · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
TO
TooManyMixRolePlay-7B-Story_V3.5
Toomanymixroleplay · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.5 9B Uncensored
Qwen · ccharnkij
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen Image 2.1 Uncensored
Qwen · Janchan123
Params7.12B
Context—
Q4_K_M4.6 GB
MI
nousresearch_yarn-mistral-7b-128k
Mistral · TheBloke
Params7.24B
Context128K
Q4_K_M4.4 GB
WE
WeVisOne VL 4B
Wevisone · mradermacher
Params4.42B
Context256K
Q4_K_M2.7 GB
NO
Noema 1.5 4B Fast
Noema · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
MI
TowerInstruct Mistral 7B v0.2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
TO
ToriiGate 0.5
Toriigate · DraconicDragon
Params4.84B
Context256K
Q4_K_M3.1 GB
GL
glm-4-9b-chat
GLM · legraphista
Params9.4B
Context128K
Smallest3.1 GB
QW
Qwen Image 2.1 Text Encoder Heretic
Qwen · chfm
Params8.2B
Context256K
Q4_K_M5.0 GB
DA
danube3-4B
Danube · h2oai
Params3.96B
Context8K
Q4_K_M2.4 GB
CH
UnCanny Photorealism Chroma
Chroma · dawncreates
Params8.9B
Context—
Q4_K_M5.6 GB
LL
Humanish Roleplay Llama 3.1 8B i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
NE
Neuralgamma 2.5 7B slerp
Neuralgamma · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
QW
Qwen3.8 4B Agentic Distill
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
FR
FrogNano 4B 2609
Frognano · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB