Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
YI
Mahou-1.3-yi-9B
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
AD
AdaGuard 8B
Adaguard · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
MI
Ministral-3-8B-Nymphaea-RP
Mistral · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
MI
Mistral 7B Instruct v0.2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
AR
Arcee-Agent
Arcee · crusoeai
Params7.62B
Context128K
Q4_K_M4.7 GB
YU
YuE2 3B Ruach Studio IRA Merge
Yue · goldhub
Params3.58B
Context—
Smallest2.9 GB
AY
aya-23-8B
Aya · legraphista
Params8B
Context8K
Smallest2.2 GB
LL
LLaMA Pro 8B
Llama · TheBloke
Params8.36B
Context4K
Q4_K_M5.1 GB
AR
ArrowPro-7B-KillerWhale
Arrowpro · mmnga
Params7.24B
Context4K
Q4_K_M4.4 GB
OL
OLMoE 1B 7B 0924 Instruct
OLMo · mradermacher
Params6.9B
Context4K
Q4_K_M4.2 GB
QW
Qwen3-8B
Qwen · ggml-org
Params8.19B
Context40K
Smallest1.1 GB
GR
Granite 3.2 8b Instruct
Granite · bartowski
Params8.17B
Context128K
Q4_K_M4.9 GB
BE
berkeley nest Starling LM 7B alpha
Berkeley · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
DA
Darwin 4B Chimera
Darwin · bartowski
Params4B
Context40K
Q4_K_M2.6 GB
PY
Pythia 6.9b
Pythia · mradermacher
Params6.86B
Context2K
Q4_K_M4.4 GB
LL
DarkIdol Llama 3.1 8B Instruct 1.0 Uncensored i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
AI
jondurbin_airoboros-m-7b-3.1.1
Airoboros · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
NO
Novaeus Promptist 7B Instruct i1
Novaeus · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
ZD
ZDTaichu5.0 9B
Zdtaichu · TaichuAI
Params8.95B
Context256K
Q4_K_M5.6 GB
IR
Irene-RP-v2-7B_128k
Irene · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LE
LemonKunoichiWizardV3
Lemonkunoichiwizardv · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
QW
Qwen2.5 7B
Qwen · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
DW
Dwight 4b
Dwight · bartowski
Params4B
Context256K
Q4_K_M2.6 GB
OR
Ornith 1.0 9B Uncensored
Ornith · PeppX
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Llama Leonida v2
Llama · mradermacher
Params8.2B
Context128K
Q4_K_M5.0 GB
WI
winter garden 7b delta
Winter · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
SZ
SzilviaB Daredevil LongWriter 8B abliterated i1
Szilviab · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
DE
Qwen3 8B DeepSeek v3.2 Speciale Distill
DeepSeek · TeichAI
Params8.19B
Context40K
Q4_K_M5.0 GB
LL
Llama 3.1 128k Dark Planet Uncensored 8B
Llama · DavidAU
Params8B
Context128K
Q4_K_M4.9 GB
MI
Mahou-1.2-mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB