Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
YI
Mahou-1.3-yi-9B
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
AD
AdaGuard 8B
Adaguard · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
GGUF
MI
Ministral-3-8B-Nymphaea-RP
Mistral · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
RoleplayTool useVision
MI
Mistral 7B Instruct v0.2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AR
Arcee-Agent
Arcee · crusoeai
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
YU
YuE2 3B Ruach Studio IRA Merge
Yue · goldhub
Params3.58B
Context—
Smallest2.9 GB
GGUF
AY
aya-23-8B
Aya · legraphista
Params8B
Context8K
Smallest2.2 GB
GGUF
LL
LLaMA Pro 8B
Llama · TheBloke
Params8.36B
Context4K
Q4_K_M5.1 GB
GGUF
AR
ArrowPro-7B-KillerWhale
Arrowpro · mmnga
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
OL
OLMoE 1B 7B 0924 Instruct
OLMo · mradermacher
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
QW
Qwen3-8B
Qwen · ggml-org
Params8.19B
Context40K
Smallest1.1 GB
GGUF
GR
Granite 3.2 8b Instruct
Granite · bartowski
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
BE
berkeley nest Starling LM 7B alpha
Berkeley · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
DA
Darwin 4B Chimera
Darwin · bartowski
Params4B
Context40K
Q4_K_M2.6 GB
Reasoning
PY
Pythia 6.9b
Pythia · mradermacher
Params6.86B
Context2K
Q4_K_M4.4 GB
GGUF
LL
DarkIdol Llama 3.1 8B Instruct 1.0 Uncensored i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
AI
jondurbin_airoboros-m-7b-3.1.1
Airoboros · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NO
Novaeus Promptist 7B Instruct i1
Novaeus · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
ZD
ZDTaichu5.0 9B
Zdtaichu · TaichuAI
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
IR
Irene-RP-v2-7B_128k
Irene · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
LE
LemonKunoichiWizardV3
Lemonkunoichiwizardv · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B
Qwen · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
Roleplay
DW
Dwight 4b
Dwight · bartowski
Params4B
Context256K
Q4_K_M2.6 GB
GGUF
OR
Ornith 1.0 9B Uncensored
Ornith · PeppX
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
LL
Llama Leonida v2
Llama · mradermacher
Params8.2B
Context128K
Q4_K_M5.0 GB
GGUF
WI
winter garden 7b delta
Winter · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
SZ
SzilviaB Daredevil LongWriter 8B abliterated i1
Szilviab · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
DE
Qwen3 8B DeepSeek v3.2 Speciale Distill
DeepSeek · TeichAI
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
LL
Llama 3.1 128k Dark Planet Uncensored 8B
Llama · DavidAU
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
MI
Mahou-1.2-mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other sizes

Keep browsing