Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
RA RFT Qwen2.5 VL 7B
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
YI
01 ai Yi 9B 200K
Yi · RichardErkhov
Params8.83B
Context256K
Q4_K_M5.3 GB
GGUF
LA
lama3 8b hikikomori v0.3
Lama · hikikomoriHaven
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GE
Gemma-3N-E4B-It
Gemma · unsloth
Params6.9B
Context32K
Q4_K_M4.5 GB
Vision
QW
ColdBrew Qwen3.5 CharThink Base
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
QW
RuntimeLabs Kayak Qwen3 VL 4B
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
Vision
MP
mpt 7b i1
Mpt · mradermacher
Params6.65B
Context2K
Q4_K_M4.3 GB
GGUF
SA
Salamandra 7b Instruct
Salamandra · RichardErkhov
Params7.77B
Context8K
Q4_K_M4.9 GB
GGUF
CO
ContextualKunoichi KTO 7B i1
Contextualkunoichi · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LF
LFM2.5 8B A1B Uncensored
Lfm · zaakirio
Params8.47B
Context125K
Q4_K_M5.2 GB
GGUF
DE
DeepHat V1 7B imatrix
Deephat · liodon-ai
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
ZE
zephyr 7b hh dpo i1
Zephyr · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 9B Uncensored Brainstorm Alpha
Llama · DavidAU
Params8.94B
Context128K
Q4_K_M5.4 GB
Roleplay
ST
starcoder2-7b
StarCoder · second-state
Params7.17B
Context16K
Q4_K_M4.4 GB
Coding
AM
llm360_amber
Amber · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
LL
lightblue suzume llama 3 8B japanese
Llama · mmnga
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
NE
Nemotron 3.5 Bolt E4B Heretic
Nemotron · mradermacher
Params7.52B
Context64K
Q4_K_M5.3 GB
ReasoningTool use
GE
Gemma 3 4b It Abliterated
Gemma · mlabonne
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
NI
Nitral AI Eris PrimeV4.20 Vision 32k 7B
Nitral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Vision
UM
Umt5 Xxl
Umt · chatpig
Params5.68B
Context512
Q4_K_M3.7 GB
Coding
SI
SIRI 7B High
Siri · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
GE
Gemma 4 E4B It OBLITERATED
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
GGUF
QW
Qwen3.8 Flash Next MTP Strix Halo
Qwen · EasiiX
Params3.88B
Context256K
Smallest4.1 GB
GGUF
QW
Qwen3.5 9B Q4 K M
Qwen · Canfield
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
LL
Llama 3.1 Minitron 4B Width Base
Llama · bartowski
Params4.51B
Context128K
Q4_K_M2.8 GB
GGUF
RU
rudra i1
Rudra · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
Tool use
DE
deepseek-coder-6.7B-kexer
DeepSeek · lmstudio-community
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
ER
ERNIE Image
Ernie · unsloth
Params8.03B
Context—
Q4_K_M5.0 GB
GGUF
AP
Apertus 8B Instruct 2509
Apertus · bartowski
Params8.05B
Context64K
Q4_K_M5.1 GB
GGUF
LL
dolphin 2.9 llama3 8b GGUF smashed
Llama · PrunaAI
Params8B
Context8K
Q4_K_M4.9 GB
GGUF

Other sizes

Keep browsing