Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
NE
NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash
Nemotron · ggml-org
Params7.52B
Context128K
Smallest0.6 GB
ReasoningTool use
QW
Qwen Image 2.1 Uncensored
Qwen · daddysteven
Params7.12B
Context—
Q4_K_M4.6 GB
GGUF
BI
BioMistral 7B
Biomistral · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
EX
EXAONE 3.5 7.8B Instruct
Exaone · LGAI-EXAONE
Params7.8B
Context32K
Q4_K_M4.8 GB
GGUF
QW
Qwen3.5 9B
Qwen · Smoffyy
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
QW
Qwen3 Jan v1 256k Ctx 6B Brainstorm20x
Qwen · mradermacher
Params5.94B
Context256K
Q4_K_M3.6 GB
Coding
MI
Mistral 7B v0.1
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
JI
JiuZhou
Jiuzhou · mradermacher
Params7.5B
Context32K
Q4_K_M4.5 GB
GGUF
AL
AlphaMonarch 7B i1
Alphamonarch · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
dolphin 2.9 llama3 8b 256k GGUF smashed
Llama · PrunaAI
Params8B
Context512K
Q4_K_M4.9 GB
GGUF
LL
CodeLlama 7b Instruct Solidity i1
Llama · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
QW
Qwen3.5 9B The Defiant Fable DARK ROAST Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningRoleplay
CO
CodeQwen1.5-7B-Chat
Codeqwen · bartowski
Params7.25B
Context64K
Q4_K_M4.7 GB
GGUF
GR
granite 4.2 8b heretic
Granite · saidutta69
Params8.79B
Context128K
Q4_K_M5.3 GB
ReasoningRoleplayTool use
GE
Gemma 4 E4B It Uncensored
Gemma · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool use
MI
OpenHermes 2.5 Mistral 7B Orca DPO
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
GL
Glm Oss 8b
GLM · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
FL
d3-flash Artemis 4.6GB
Flash · turintech
Params8.95B
Context256K
Smallest4.6 GB
ReasoningTool use
QW
Josiefied Qwen3 8B Abliterated v1
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
QW
Qwen3.5 9b Triple Loop Fase2
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
LL
Llama 3 ELYZA JP 8B
Llama · elyza
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
AS
Aspire 8B model stock i1
Aspire · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Heretic Qwen3.5 4b
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
TU
Tulu2 7b_writing Prompts
Tulu · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
danielbrdz_barcenas-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
rana shahroz mistral 7b openorca20k
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IM
z image solordz v2
Image · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
GGUF
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
GR
Granite 8b Code Base 4k
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
Coding

Other sizes

Keep browsing