Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
MI
Kant-Test-0.1-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
BE
BeagleNuBuRPInfinWestLakev2 IreneRP Neural 7B slerp
Beaglenuburpinfinwestlakev · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
QW
Qwen2.5 7B Comment Sexuality Reasoner
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
BA
BanglaLLama-3-8b-BnWiki-Base
Banglallama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
XL
xLAM 7b R
Xlam · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 3b NSFW Aesir Uncensored
Llama · Novaciano
Params3.61B
Context128K
Smallest2.0 GB
ST
Starling LM 7B Beta
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
NI
Ninja v1 NSFW 128k
Ninja · mmnga
Params7.24B
Context128K
Q4_K_M4.4 GB
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
Gemma 4 E2B it qat NVFP4
Gemma · WhiskyAKM
Params4.63B
Context128K
Smallest0.1 GB
IR
iris 7b
Iris · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
SW
Swallow 7B Instruct
Swallow · TheBloke
Params6.83B
Context4K
Q4_K_M4.1 GB
QW
Parable Qwen3 8B Claude Fable 5
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
MI
microsoft_WizardLM-2-7B
Microsoft · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 3B Instruct Abliterated
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
WI
WizardLM 2 7B GGUF smashed
Wizardlm · PrunaAI
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Open Insurance LLM Llama3 8B
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
LL
Llama Krikri 8b Instruct
Llama · ilsp
Params8.2B
Context128K
Q4_K_M5.0 GB
MI
Mistral 7b Sandbagging Cb New
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TW
tweety 7b tatar v24a
Tweety · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mixtral AI DeepMedicalMind
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
IC
darkps ice AI
Ice · bartowski
Params8.2B
Context40K
Q4_K_M5.2 GB
ID
ideogram 4
Ideogram · molbal
Params9.28B
Context—
Smallest5.6 GB
CG
cgato TheSpice 7b FT ExperimentalOrca
Cgato · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mr Grammatology clinical problems Mistral 7B 0.5
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
RD
RDT2 VQ
Rdt · mradermacher
Params7.6B
Context125K
Q4_K_M4.7 GB
MU
music generation model
Music · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
RE
ReflectionCoder-DS-6.7B
Reflectioncoder · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
DE
Andy 3.5
DeepSeek · Sweaterdog
Params7.62B
Context128K
Q4_K_M1.1 GB
QW
Qwen3.8 Flash Next MTP
Qwen · Joakimpalm-Zen
Params3.88B
Context256K
Smallest4.2 GB