Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3.5 9B DS9 USS Defiant
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Llama3.1 8B PlumMath
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
LL
Discoleo
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
YA
models--automerger--Yamshadowexperiment28T3qm7xp-7B
Yamshadowexperiment · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 Esper3 Reasoning Instruct 6B Brainstorm20x Enhanced E32 128k Ctx
Qwen · mradermacher
Params5.94B
Context128K
Q4_K_M3.6 GB
CO
Codegeex4 All 9b
Codegeex · mradermacher
Params9.4B
Context128K
Q4_K_M6.3 GB
KU
KukulStanta InfinityRP 7B slerp
Kukulstanta · AlekseiPravdin
Params7.24B
Context32K
Q4_K_M4.4 GB
VI
vicgalle Configurable Janus 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
WA
Wan2.2 TI2V 5B Turbo
Wan · Kiijoku
Params5B
Context—
Q4_K_M3.4 GB
LL
Meta-Llama-3-8B-Instruct
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
MI
NSFW DPO Noromaid 7b Mistral 7B Instruct v0.1
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
LL
ADG CoT LLaMa3 8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
MU
G4 E4B Musica v1
Musica · bartowski
Params7.5B
Context128K
Q4_K_M5.4 GB
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · wangkanai
Params8.19B
Context256K
Smallest5.0 GB
JA
JarvisArt 1208
Jarvisart · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
LW
models--mergekit-community--mergekit-slerp-ozbsicw
Lwm · MaziyarPanahi
Params6.74B
Context1024K
Q4_K_M4.1 GB
QW
Qwen3.5 4B
Qwen · prithivMLmods
Params4.33B
Context256K
Q4_K_M2.8 GB
LL
Meta Llama 3.1 8B Instruct Heretic
Llama · saidutta69
Params8.03B
Context128K
Q4_K_M4.9 GB
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
LL
Meta-Llama-3-8B-Instruct-abliterated-v3
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen3.5 9B Uncensored
Qwen · xCloudinfo
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Llama 3 8B TKK Elite V1.0
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GE
new adult writer
Gemma · ahmed22xa
Params7.52B
Context128K
Q4_K_M8.6 GB
MI
MiniCPM O 2_6
MiniCPM · bartowski
Params7.6B
Context32K
Q4_K_M4.7 GB
LL
TransLLaMA2-7B
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
LU
Ko-Luxia-8B-it
Luxia · mradermacher
Params8.17B
Context8K
Q4_K_M5.0 GB
ST
stable diffusion 3.5 large turbo
Stable · city96
Params8.15B
Context—
Smallest4.8 GB
LL
FuseChat Llama 3.1 8B Instruct
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
AN
Andy 4.2
Andy · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Llama-3-8B-Magpie-Mix-SFT-v0.1
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB