Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
CL
Clef-Flash
Clef · turintech
Params9.08B
Context256K
Smallest3.6 GB
ReasoningTool use
AU
AuraFlow v0.3
Auraflow · city96
Params6.86B
Context—
Q4_K_M4.0 GB
GGUF
HU
Huihui Ornith 1.5 9B Abliterated
Huihui · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
MI
Mahou-1.3a-mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Roleplay-Mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
QW
flux2 klein 4B uncensored text encoder
Qwen · WeReCooking
Params4B
Context40K
Smallest2.4 GB
Coding
MI
cognitivecomputations_dolphin-2.6-mistral-7b-dpo-laser
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
CodingTool use
HU
Huihui Hunyuan MT 7B Abliterated
Huihui · mradermacher
Params7.5B
Context256K
Q4_K_M4.6 GB
GGUF
QW
Qwen3 4B Hivemind Instruct Heretic Abliterated Uncensored NEO Imatrix
Qwen · DavidAU
Params4B
Context256K
Q4_K_M2.5 GB
Roleplay
KE
Kev-9B Artemis 4.1GB
Kev · turintech
Params8.96B
Context256K
Smallest4.1 GB
ReasoningTool use
JA
Jan-v1-4B
Jan · janhq
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
KA
Kainaticulous Rp 7b
Kainaticulous · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
QW
Qwen3 8B JP Uncensored
Qwen · ryo559
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
GE
gemma-2-9b-it
Gemma · legraphista
Params9.24B
Context8K
Smallest2.4 GB
GGUF
LL
togethercomputer_llama-2-7b-32k-instruct
Llama · TheBloke
Params6.74B
Context32K
Q4_K_M4.1 GB
GGUF
TA
UI TARS 7B DPO
Tars · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
Vision
MI
Sydney_Pirate_Mistral_7b
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ME
q-bert_metamath-cybertron-starling
Metamath · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FL
Sakura FLUX.3 Image Klein PromptUpsampler Uncensored
Flux · webmp3
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
QW
Qwen3 4B toolcalling gguf codex
Qwen · Manojb
Params4B
Context256K
Smallest4.3 GB
CodingTool use
QW
Qwen3.5 4B
Qwen · dahara1
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool useVision
NE
NVIDIA Nemotron 3.5 Bolt E4B
Nemotron · bartowski
Params7.52B
Context64K
Q4_K_M5.2 GB
ReasoningTool useVision
DI
digital socrates 7b i1
Digital · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GE
Mahou-1.3-gemma2-9B
Gemma · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
DE
DAN L3 R1 8B
DeepSeek · UnfilteredAI
Params8B
Context128K
Smallest16.1 GB
Reasoning
PE
PentaCoder 9B
Pentacoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoning
FL
Flux2 Klein 9b Uncensored Text Encoder
Flux · wy8edhuh
Params8.19B
Context40K
Smallest5.9 GB
Coding
GE
Gemma 4 E2B It Uncensored MAX
Gemma · prithivMLmods
Params4.65B
Context128K
Smallest5.0 GB
ReasoningVision
QW
security 8b
Qwen · bebrws
Params8.2B
Context128K
Q4_K_M5.0 GB
Tool use
GR
Granite 8b Code Base 128k
Granite · RichardErkhov
Params8.05B
Context125K
Q4_K_M4.9 GB
Coding

Other sizes

Keep browsing