Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
SC
Scout 8b
Scout · mradermacher
Params8.31B
Context32K
Q4_K_M5.1 GB
ReasoningRoleplay
YI
EVA Yi 1.5 9B 32K V1
Yi · mradermacher
Params8.83B
Context32K
Q4_K_M5.3 GB
GGUF
LL
Hermes 2 Pro Llama 3 8B
Llama · AlekseiPravdin
Params7.81B
Context8K
Q4_K_M4.8 GB
Tool use
DE
Qwen3.5 9B DeepSeek V4 Flash
DeepSeek · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
MI
Mahou-1.2a-mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BA
jondurbin_bagel-dpo-7b-v0.1
Bagel · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
SC
Scout 4b
Scout · mradermacher
Params3.88B
Context128K
Q4_K_M2.5 GB
ReasoningRoleplay
QW
Qwen3.5 4B Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
NE
mlabonne_neuralpipe-7b-ties
Neuralpipe · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LI
Ling 3.0 Tiny Uncensored Abliterated
Ling · mradermacher
Params7.89B
Context128K
Q4_K_M4.8 GB
Reasoning
VE
Veena
Veena · Mungert
Params3.78B
Context128K
Q4_K_M2.5 GB
GGUF
MO
models--automerger--Yamshadowexperiment28M7-7B
MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
AceReason Nemotron 1.1 7B
Nemotron · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
CodingReasoning
SE
SenseNova U1.5 8B MoT
Sensenova · mradermacher
Params9.35B
Context256K
Q4_K_M5.7 GB
ReasoningTool use
VI
VisCoder 7B
Viscoder · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
QW
Qwen3 4B Instruct 2507 heretic
Qwen · saidutta69
Params4B
Context256K
Q4_K_M2.5 GB
ReasoningRoleplayTool use
SP
Spark X2.6 4B
Spark · prithivMLmods
Params4.11B
Context1024K
Q4_K_M2.6 GB
ReasoningTool use
DA
Daschund
Daschund · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
Tool use
NU
Nuqta Arabic OCR 4B
Nuqta · SalahALHaismawi
Params4.54B
Context256K
Q4_K_M2.9 GB
ReasoningTool useVision
SW
Swallow MS 7b v0.1
Swallow · mradermacher
Params7.33B
Context4K
Q4_K_M4.4 GB
GGUF
LE
LegalOne 8B
Legalone · mradermacher
Params8.2B
Context40K
Q4_K_M5.0 GB
GGUF
QW
Qwen3.5 9B Claude 4.6 OS HERETIC UNCENSORED INSTRUCT
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Roleplay
OL
OLMo 2 1124 7B Instruct
OLMo · allenai
Params7.3B
Context4K
Q4_K_M4.5 GB
GGUF
IN
InternLM2
InternLM · bartowski
Params7.74B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
LI
Liquid A1B 8B PL Coder
Liquid · mradermacher
Params8.34B
Context125K
Q4_K_M5.0 GB
Coding
MA
Marcoroni 7b
Marcoroni · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Fireball Alpaca Llama3.1.08 8B Philos C R1 i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Reasoning
QW
Qwen3.8 9B Heretic Uncensored
Qwen · marafx2007
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
GR
Granite 8b Code Instruct 128k
Granite · RichardErkhov
Params8.05B
Context125K
Q4_K_M4.9 GB
Coding

Other sizes

Keep browsing