Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Kant-Test-0.1-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BE
BeagleNuBuRPInfinWestLakev2 IreneRP Neural 7B slerp
Beaglenuburpinfinwestlakev · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Qwen2.5 7B Comment Sexuality Reasoner
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Reasoning
BA
BanglaLLama-3-8b-BnWiki-Base
Banglallama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
XL
xLAM 7b R
Xlam · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Llama 3.2 3b NSFW Aesir Uncensored
Llama · Novaciano
Params3.61B
Context128K
Smallest2.0 GB
Roleplay
ST
Starling LM 7B Beta
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
NI
Ninja v1 NSFW 128k
Ninja · mmnga
Params7.24B
Context128K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
GE
Gemma 4 E2B it qat NVFP4
Gemma · WhiskyAKM
Params4.63B
Context128K
Smallest0.1 GB
Vision
IR
iris 7b
Iris · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
SW
Swallow 7B Instruct
Swallow · TheBloke
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Parable Qwen3 8B Claude Fable 5
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
Reasoning
MI
microsoft_WizardLM-2-7B
Microsoft · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 3B Instruct Abliterated
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
GGUF
WI
WizardLM 2 7B GGUF smashed
Wizardlm · PrunaAI
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Open Insurance LLM Llama3 8B
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama Krikri 8b Instruct
Llama · ilsp
Params8.2B
Context128K
Q4_K_M5.0 GB
GGUF
MI
Mistral 7b Sandbagging Cb New
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TW
tweety 7b tatar v24a
Tweety · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mixtral AI DeepMedicalMind
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
IC
darkps ice AI
Ice · bartowski
Params8.2B
Context40K
Q4_K_M5.2 GB
GGUF
ID
ideogram 4
Ideogram · molbal
Params9.28B
Context—
Smallest5.6 GB
GGUF
CG
cgato TheSpice 7b FT ExperimentalOrca
Cgato · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mr Grammatology clinical problems Mistral 7B 0.5
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
RD
RDT2 VQ
Rdt · mradermacher
Params7.6B
Context125K
Q4_K_M4.7 GB
Vision
MU
music generation model
Music · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RE
ReflectionCoder-DS-6.7B
Reflectioncoder · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
DE
Andy 3.5
DeepSeek · Sweaterdog
Params7.62B
Context128K
Q4_K_M1.1 GB
Reasoning
QW
Qwen3.8 Flash Next MTP
Qwen · Joakimpalm-Zen
Params3.88B
Context256K
Smallest4.2 GB
ReasoningTool use

Other sizes

Keep browsing