Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5 9B DS9 USS Defiant
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Roleplay
LL
Llama3.1 8B PlumMath
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Discoleo
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
YA
models--automerger--Yamshadowexperiment28T3qm7xp-7B
Yamshadowexperiment · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 Esper3 Reasoning Instruct 6B Brainstorm20x Enhanced E32 128k Ctx
Qwen · mradermacher
Params5.94B
Context128K
Q4_K_M3.6 GB
CodingReasoning
CO
Codegeex4 All 9b
Codegeex · mradermacher
Params9.4B
Context128K
Q4_K_M6.3 GB
GGUF
KU
KukulStanta InfinityRP 7B slerp
Kukulstanta · AlekseiPravdin
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
VI
vicgalle Configurable Janus 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
WA
Wan2.2 TI2V 5B Turbo
Wan · Kiijoku
Params5B
Context—
Q4_K_M3.4 GB
GGUF
LL
Meta-Llama-3-8B-Instruct
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
NSFW DPO Noromaid 7b Mistral 7B Instruct v0.1
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
Roleplay
LL
ADG CoT LLaMa3 8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MU
G4 E4B Musica v1
Musica · bartowski
Params7.5B
Context128K
Q4_K_M5.4 GB
ReasoningRoleplayVision
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · wangkanai
Params8.19B
Context256K
Smallest5.0 GB
Vision
JA
JarvisArt 1208
Jarvisart · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
LW
models--mergekit-community--mergekit-slerp-ozbsicw
Lwm · MaziyarPanahi
Params6.74B
Context1024K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.5 4B
Qwen · prithivMLmods
Params4.33B
Context256K
Q4_K_M2.8 GB
Vision
LL
Meta Llama 3.1 8B Instruct Heretic
Llama · saidutta69
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
LL
Meta-Llama-3-8B-Instruct-abliterated-v3
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.5 9B Uncensored
Qwen · xCloudinfo
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
LL
Llama 3 8B TKK Elite V1.0
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GE
new adult writer
Gemma · ahmed22xa
Params7.52B
Context128K
Q4_K_M8.6 GB
RoleplayVision
MI
MiniCPM O 2_6
MiniCPM · bartowski
Params7.6B
Context32K
Q4_K_M4.7 GB
CodingVision
LL
TransLLaMA2-7B
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LU
Ko-Luxia-8B-it
Luxia · mradermacher
Params8.17B
Context8K
Q4_K_M5.0 GB
GGUF
ST
stable diffusion 3.5 large turbo
Stable · city96
Params8.15B
Context—
Smallest4.8 GB
GGUF
LL
FuseChat Llama 3.1 8B Instruct
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
AN
Andy 4.2
Andy · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
LL
Llama-3-8B-Magpie-Mix-SFT-v0.1
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF

Other sizes

Keep browsing