Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama Guard 3 8B
Llama · legraphista
Params8B
Context128K
Smallest2.0 GB
GGUF
DO
Dolphin X1 8B
Dolphin · dphn
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GL
AutoGLM Phone 9B
GLM · Luckybalabala
Params9.4B
Context64K
Q4_K_M6.2 GB
Vision
DA
DataPilot ArrowPro 7B KUJIRA
Datapilot · mmnga
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
IM
Z Image Turbo
Image · vantagewithai
Params6.15B
Context—
Q4_K_M5.0 GB
GGUF
ST
L3-8B-Stheno-v3.3-32K
Stheno · DavidAU
Params8B
Context32K
Q4_K_M4.9 GB
Roleplay
ST
Starling-LM-7B-alpha-x2
Starling · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
CO
C4Ai Command R7B 12 2024
Command R · bartowski
Params8B
Context8K
Q4_K_M5.1 GB
GGUF
MI
E5 Mistral 7B Instruct Embedding
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
Embedding
GR
Granite 4.0 H Tiny
Granite · bartowski
Params6.94B
Context1024K
Q4_K_M4.3 GB
GGUF
QW
Qwen2.5 7B Instruct Heretic
Qwen · saidutta69
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
LL
dolphin 2.9 llama3 8b
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
CodingTool use
GE
Gemma-4-E4B-It
Gemma · ente-ai
Params7.5B
Context128K
Q4_K_M5.0 GB
Vision
AS
AstaBrief_8B
Astabrief · mradermacher
Params8.2B
Context40K
Q4_K_M5.0 GB
GGUF
SP
Spark X2.6 4B Uncensored GSQ RCO
Spark · abenzerps
Params4.11B
Context1024K
Smallest1.6 GB
ReasoningTool use
QW
Qwen3.8 4B Heretic
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
GE
Gemma 4 E4B It Ultra Uncensored Heretic
Gemma · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
QW
Qwen3 8B Socratic SFT
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
QW
Crow 9B HERETIC 4.6
Qwen · Crownelius
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningVision
GE
Skaldic Gemma 4 E4B
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
GGUF
AI
jondurbin_airoboros-m-7b-3.1
Airoboros · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
JA
Jais-3-7B
Jais · mradermacher
Params9B
Context512K
Q4_K_M5.6 GB
Reasoning
QW
Wrench 9B Q4 K M
Qwen · ClankLabs
Params8B
Context256K
Q4_K_M5.6 GB
Tool use
UM
Umievo-itr012-Gleipnir-7B
Umievo · mmnga
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B Instruct Uncensored
Qwen · WSDW
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
GE
Gemma 4 E4B It Apostate
Gemma · mradermacher
Params8.13B
Context128K
Q4_K_M5.0 GB
ReasoningTool use
QI
QiMing Holos Plus 8B
Qiming · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
samantha-dolphin-llama-3
Llama · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MY
MythoLogic Mini 7B
Mythologic · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
YO
Youri 7b Instruction
Youri · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF

Other sizes

Keep browsing