Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LU
L3-8B-Lunaris-v1
Lunaris · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
teknium_openhermes-2.5-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
VE
Veloxy Flash 15B i1
Veloxy · zveloxy
Params8.95B
Context512K
Q4_K_M5.6 GB
Reasoning
GE
Gemma 4 E4B It OBLITERATED v3.1
Gemma · Aimeeluvssya
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
LL
Dolphin3.0 Llama3.1 8B Abliterated
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
PH
Phi-4-Mini-Reasoning
Phi · unsloth
Params3.84B
Context128K
Q4_K_M2.5 GB
CodingReasoning
ST
sd3.5 large
Stable · calcuis
Params8.15B
Context—
Smallest4.8 GB
GGUF
QW
Qwen2.5 VL 7B NSFW Caption V3
Qwen · bartowski
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
PH
Phi 3 mini 4k instruct GGUF Imatrix smashed
Phi · PrunaAI
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
QW
Qwen2.5 7B Instruct Abliterated
Qwen · richardyoung
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Lexi-Llama-3-8B-Uncensored
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwopus3.5 9B Coder Templatefix Hf
Qwopus · Jackrong
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningTool use
ST
StartLux Decision v1 4B Q8 0
Startlux · startlux-models
Params4.21B
Context256K
Smallest4.5 GB
GGUF
LI
LightOnOCR 3 4B
Lightonocr · prithivMLmods
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
DE
decider 4b
Decider · Mapika
Params4.2B
Context256K
Q4_K_M2.7 GB
GGUF
QW
Qwen3.8 9B Distill Uncensored Heretic
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · noctrex
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
TR
Translategemma 4b It
Translategemma · SandLogicTechnologies
Params3.88B
Context128K
Q4_K_M2.5 GB
GGUF
LY
Lythri 4B A2B
Lythri · bartowski
Params4.63B
Context128K
Q4_K_M3.2 GB
GGUF
NE
NeoHorse-1-4B
Neohorse · TokenRhythm
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
DE
WhiteRabbitNeo WhiteRabbitNeo V3 7B
Deephat · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
QW
Qwen3 VL 8B Heretic Stable
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
GL
Glm-4-9B-0414
GLM · unsloth
Params9.4B
Context32K
Q4_K_M6.2 GB
GGUF
FO
Foundation Sec 8B Reasoning
Foundation · fdtn-ai
Params8B
Context128K
Q4_K_M4.9 GB
Reasoning
MI
minimax_music3_dit_Q6_K
Minimax · ChrisColeTech
Params7.61B
Context—
Smallest2.1 GB
GGUF
QW
Qwen3 8B Heretic
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
QW
Qwen3.5 9B Coder
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningTool use
NE
NVIDIA Nemotron 3.5 Bolt E4B Uncensored
Nemotron · AtomicChat
Params7.5B
Context64K
Q4_K_M5.1 GB
GGUF
FL
FLUX.2 klein base 4B
Flux · leejet
Params3.88B
Context—
Smallest2.5 GB
GGUF
MA
UI Mate 9B
Mate · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Vision

Other sizes

Keep browsing