Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
FL
flux2 klein
Flux · ChrisColeTech
Params9.08B
Context40K
Q4_K_M5.0 GB
Vision
QW
Sakura FLUX.3 Image Klein TextEncoder Uncensored
Qwen · webmp3
Params4B
Context256K
Q4_K_M2.5 GB
CodingVision
OL
OLMoE 1B 7B 0924 Instruct
OLMo · mradermacher
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
NO
normistral 7b warm
Normistral · ltg
Params7.25B
Context2K
Q4_K_M4.4 GB
Coding
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · Orenguteng
Params8B
Context128K
Smallest4.7 GB
GGUF
JI
JiRackUltra 7b
Jirackultra · CMSManhattan
Params7.62B
Context128K
Q4_K_M4.7 GB
ReasoningTool use
QW
Qwen2.5 Coder 7B Instruct Uncensored
Qwen · Fbjfhvgv
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
SP
Spark X2.5 4B uncensored
Spark · InfinimindCreations
Params4.1B
Context1024K
Q4_K_M2.6 GB
GGUF
GE
gemma 4 E4B it heretic QAT
Gemma · SC117
Params7.46B
Context128K
Smallest4.3 GB
CodingVision
GE
Gemma-3N-E2B-It
Gemma · unsloth
Params4.46B
Context32K
Q4_K_M3.0 GB
Vision
RA
RakutenAI 7B Chat
Rakutenai · mradermacher
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Qwen2.5-VL-7B-Instruct
Qwen · Mungert
Params7.6B
Context125K
Q4_K_M4.8 GB
Vision
LL
Llama-3-Groq-8B-Tool-Use
Llama · lmstudio-community
Params8B
Context8K
Q4_K_M4.9 GB
Tool use
QW
Qwen3.5 4B MultiDistill 3xMerge UNCENSORED Heretic
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
Reasoning
WA
Wan2.2 Fun 5B InP
Wan · QuantStack
Params5B
Context—
Q4_K_M3.4 GB
GGUF
FL
flux 3 image klein prompt upsampler
Flux · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
FR
FrogNano 4B Heretic
Frognano · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
CodingRoleplayTool use
YI
01-ai_yi-6b-200k
Yi · TheBloke
Params6.06B
Context195K
Q4_K_M3.7 GB
GGUF
OX
OxCoder 9B
Oxcoder · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingVision
QW
Qwen Image 2.1
Qwen · molbal
Params7.12B
Context—
Smallest4.2 GB
GGUF
GR
granite-8b-code-instruct
Granite · mradermacher
Params8.05B
Context4K
Q4_K_M4.9 GB
CodingTool use
GE
Gemmasutra Small 4B v1
Gemma · TheDrummer
Params3.86B
Context8K
Q4_K_M2.5 GB
GGUF
GE
Gemma 4 E4B it Artemis
Gemma · turintech
Params7.46B
Context128K
Smallest0.1 GB
ReasoningTool useVision
QW
Qwen2.5 VL 7B Instruct
Qwen · DhruvalLabs
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
YI
Yi-1.5-9B-Chat
Yi · bartowski
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
QW
Qwen3-8B-128K
Qwen · unsloth
Params8.19B
Context128K
Q4_K_M5.0 GB
GGUF
OR
Ornith1.0 9B 3MPER0RR Abliterated
Ornith · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
QW
Qwen3.8 9b Hermes Function Calling v2
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
Tool useVision
OL
Olmo-3-7B-Think
OLMo · unsloth
Params7.3B
Context64K
Q4_K_M4.5 GB
Reasoning
GL
Glm 4 9b Chat
GLM · bartowski
Params9.4B
Context128K
Q4_K_M6.3 GB
GGUF

Other sizes

Keep browsing