Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
GR
Granite 3.3 8b Instruct
Granite · mrutkows
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
CO
ComfyUI Starter Packs
Comfyui · MaxedOut
Params4.76B
Context—
Q4_K_M9.7 GB
GGUF
QW
qwen3.5 9b null space abliterated
Qwen · jwest33
Params8.95B
Context256K
Smallest7.4 GB
Vision
IN
InSight Doc 8B
Insight · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Tool useVision
QW
Qwen3 4B Instruct 2507
Qwen · a-ivanovitch
Params4B
Context256K
Q4_K_M2.5 GB
Tool use
PE
Peach 2.0 9B 8k Roleplay
Peach · mradermacher
Params8.83B
Context16K
Q4_K_M5.3 GB
Roleplay
LL
Llama 3.1 8B MultiReflection Instruct i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GL
LongWriter Glm4 9b
GLM · QuantFactory
Params9.4B
Context1024K
Q4_K_M6.3 GB
GGUF
IM
POCKET Zimage CPU
Image · FINAL-Bench
Params6.15B
Context—
Smallest3.5 GB
GGUF
MI
dolphin-2.9.3-mistral-7B-32k
Mistral · bartowski
Params7.25B
Context32K
Q4_K_M4.4 GB
CodingTool use
ST
Starling LM 7B alpha ExPO
Starling · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
GE
Amoral Gemma3 4B
Gemma · bartowski
Params3.88B
Context128K
Q4_K_M2.5 GB
GGUF
KR
Kronumos
Kronumos · mradermacher
Params7.6B
Context32K
Q4_K_M4.7 GB
Coding
BA
bagel 8b v1.0 i1
Bagel · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
CodingRoleplayTool use
QW
Qwen3.8 9B
Qwen · hakergeniusz
Params9.2B
Context256K
Q4_K_M5.8 GB
GGUF
QW
Qwen 3 VL 8B Instruct Heretic
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Tool useVision
GE
Gemma 2 9b It
Gemma · duyntnet
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
Llama2 7B Hindi finetuned
Llama · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB
Coding
MO
models--automerger--Inex12T3qm7xp-7B
MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TE
Tema_Q R2.0 Code
Tema · mradermacher
Params8.5B
Context8K
Q4_K_M5.3 GB
Coding
LL
Hermes-2-Theta-Llama-3-8B-32k
Llama · mradermacher
Params8B
Context32K
Q4_K_M4.9 GB
Tool use
GE
Baihua Gemma4 E4B
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
Vision
SA
SauerkrautLM 7b LaserChat
Sauerkrautlm · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
RO
L3.2 Rogue Creative Instruct 7B
Rogue · DavidAU
Params7.14B
Context128K
Q4_K_M4.4 GB
Roleplay
LL
deepthought 8b llama v0.01 alpha
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.5 9B The Bradbury F451 Pro Writer Uncensored Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingRoleplay
MI
arc53_docsgpt-7b-mistral
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CH
ChatLaw2E_plain_7B
Chatlaw · mradermacher
Params7.74B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Llama-3.3-8B-Instruct
Llama · Mungert
Params8B
Context8K
Q4_K_M5.3 GB
GGUF
DO
Dolphin3 Cyber 8B
Dolphin · ScottzillaSystems
Params8B
Context128K
Q4_K_M4.9 GB
GGUF

Other sizes

Keep browsing