Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
RA RFT Qwen2.5 VL 7B
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
YI
01 ai Yi 9B 200K
Yi · RichardErkhov
Params8.83B
Context256K
Q4_K_M5.3 GB
LA
lama3 8b hikikomori v0.3
Lama · hikikomoriHaven
Params8B
Context8K
Q4_K_M4.9 GB
GE
Gemma-3N-E4B-It
Gemma · unsloth
Params6.9B
Context32K
Q4_K_M4.5 GB
QW
ColdBrew Qwen3.5 CharThink Base
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
RuntimeLabs Kayak Qwen3 VL 4B
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
MP
mpt 7b i1
Mpt · mradermacher
Params6.65B
Context2K
Q4_K_M4.3 GB
SA
Salamandra 7b Instruct
Salamandra · RichardErkhov
Params7.77B
Context8K
Q4_K_M4.9 GB
CO
ContextualKunoichi KTO 7B i1
Contextualkunoichi · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LF
LFM2.5 8B A1B Uncensored
Lfm · zaakirio
Params8.47B
Context125K
Q4_K_M5.2 GB
DE
DeepHat V1 7B imatrix
Deephat · liodon-ai
Params7.62B
Context32K
Q4_K_M4.7 GB
ZE
zephyr 7b hh dpo i1
Zephyr · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 9B Uncensored Brainstorm Alpha
Llama · DavidAU
Params8.94B
Context128K
Q4_K_M5.4 GB
ST
starcoder2-7b
StarCoder · second-state
Params7.17B
Context16K
Q4_K_M4.4 GB
AM
llm360_amber
Amber · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
LL
lightblue suzume llama 3 8B japanese
Llama · mmnga
Params8B
Context8K
Q4_K_M4.9 GB
NE
Nemotron 3.5 Bolt E4B Heretic
Nemotron · mradermacher
Params7.52B
Context64K
Q4_K_M5.3 GB
GE
Gemma 3 4b It Abliterated
Gemma · mlabonne
Params3.88B
Context128K
Q4_K_M2.5 GB
NI
Nitral AI Eris PrimeV4.20 Vision 32k 7B
Nitral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
UM
Umt5 Xxl
Umt · chatpig
Params5.68B
Context512
Q4_K_M3.7 GB
SI
SIRI 7B High
Siri · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
GE
Gemma 4 E4B It OBLITERATED
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
QW
Qwen3.8 Flash Next MTP Strix Halo
Qwen · EasiiX
Params3.88B
Context256K
Smallest4.1 GB
QW
Qwen3.5 9B Q4 K M
Qwen · Canfield
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Llama 3.1 Minitron 4B Width Base
Llama · bartowski
Params4.51B
Context128K
Q4_K_M2.8 GB
RU
rudra i1
Rudra · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
DE
deepseek-coder-6.7B-kexer
DeepSeek · lmstudio-community
Params6.74B
Context16K
Q4_K_M4.1 GB
ER
ERNIE Image
Ernie · unsloth
Params8.03B
Context—
Q4_K_M5.0 GB
AP
Apertus 8B Instruct 2509
Apertus · bartowski
Params8.05B
Context64K
Q4_K_M5.1 GB
LL
dolphin 2.9 llama3 8b GGUF smashed
Llama · PrunaAI
Params8B
Context8K
Q4_K_M4.9 GB