Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
JA
Jan code 4b
Jan · janhq
Params4.4B
Context256K
Q4_K_M2.7 GB
GE
Gemma 4 E2B It Ultra Uncensored Heretic
Gemma · llmfan46
Params4.65B
Context128K
Q4_K_M3.4 GB
GL
GLM 4 9B 0414
GLM · bartowski
Params9.4B
Context32K
Q4_K_M6.2 GB
LI
T lite it 2.1
Lite · t-tech
Params8.2B
Context40K
Q4_K_M5.0 GB
LL
Hermes 3 Llama 3.1 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen3 Reranker 4B GGUF llama cpp
Qwen · Voodisss
Params4B
Context40K
Q4_K_M2.5 GB
QW
Qwen3 VL 8B NSFW Caption V4.5
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
QW
Huihui Qwen3 4B Abliterated v2
Qwen · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
QW
Qwen2.5 7B Instruct 1M
Qwen · lmstudio-community
Params7.62B
Context986K
Q4_K_M4.7 GB
PO
pony v7 base
Pony · Runware
Params6.86B
Context—
Smallest3.9 GB
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
MI
Ministral-3-8B-Reasoning-2512
Mistral · unsloth
Params8.49B
Context256K
Q4_K_M5.2 GB
QW
Qwen2.5 VL 7B Instruct Unredacted MAX
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
FL
flux 3 image klein base
Flux · unsloth
Params5.54B
Context—
Q4_K_M3.6 GB
QW
Qwen3-4B-Thinking-2507
Qwen · unsloth
Params4B
Context256K
Q4_K_M2.5 GB
FL
flux 3 image klein sd
Flux · Abiray
Params5.55B
Context—
Q4_K_M3.3 GB
QW
Qwen3.8 9B Distill Uncensored Heretic
Qwen · petruhonk
Params9.2B
Context256K
Smallest9.8 GB
DE
DeepSeek R1 Distill Llama 8B
DeepSeek · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
OM
OmniCoder 9B
Omnicoder · Tesslate
Params8.95B
Context256K
Q4_K_M5.7 GB
QW
Qwen3.8 4B Distill Heretic Abliterated
Qwen · yachen4ever
Params4.33B
Context256K
Q4_K_M2.8 GB
GL
GLM 4.6V Flash
GLM · bartowski
Params9.4B
Context128K
Q4_K_M6.2 GB
QW
qwen image 2.1 comfy
Qwen · t8star
Params8.95B
Context256K
Q4_K_M5.6 GB
MI
MiniCPM V 2 6
MiniCPM · openbmb
Params7.61B
Context32K
Q4_K_M4.7 GB
LY
Lythri 4B A2B
Lythri · mradermacher
Params4.63B
Context128K
Q4_K_M3.4 GB
TA
UI TARS 1.5 7B
Tars · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
FL
FLUX.2 klein base 4B
Flux · unsloth
Params3.88B
Context—
Q4_K_M2.4 GB
GE
Gemma 3 4B VL It Gemini Pro Heretic Uncensored Thinking
Gemma · Andycurrent
Params3.88B
Context128K
Q4_K_M2.5 GB
GR
Granite 4.1 8b
Granite · ibm-granite
Params8.79B
Context128K
Q4_K_M5.3 GB
QW
Qwen2.5 Coder 7B Instruct Uncensored
Qwen · BlossomsAI
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
Mistral 7B Instruct v0.2 llamafile
Mistral · mozilla-ai
Params7.24B
Context32K
Smallest14.5 GB