Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
FA
Fara 7B
Fara · bartowski
Params7.62B
Context125K
Q4_K_M4.7 GB
LL
Llava v1.6 Vicuna 7B
LLaVA · second-state
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen_Qwen3 VL 8B Thinking
Qwen · lmstudio-community
Params8.19B
Context256K
Q4_K_M5.0 GB
DE
DeepSeek V4 Pro Qwen3.5 4B
DeepSeek · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
QW
nsfwvision qwen3 vl 8b v3
Qwen · GitMylo
Params8.2B
Context256K
Q4_K_M5.0 GB
QW
Qwen2.5 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
CL
ClaudioItaly Evolutionstory 7B v3.0
Claudioitaly · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
flammenai flammen15X mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 VL 7B Instruct
Qwen · second-state
Params7.62B
Context125K
Q4_K_M4.7 GB
RU
RusMind 4.5.4 Momentum
Rusmind · DICPlay
Params4.65B
Context128K
Q4_K_M3.4 GB
HU
Hunyuan MT Chimera 7B
Hunyuan · mradermacher
Params7.5B
Context256K
Q4_K_M4.6 GB
BU
Buddy E4B v2
Buddy · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
OE
Oellm 9b 128k Theta32M
Oellm · mradermacher
Params9.1B
Context128K
Q4_K_M5.7 GB
LI
Ling-3.0-tiny-base-30T
Ling · FriskyFennec
Params8.2B
Context8K
Q4_K_M5.0 GB
FL
FLUX-3-Image-Klein-pup
Flux · AtomicChat
Params8.95B
Context256K
Smallest4.7 GB
WI
WizardLM-2-7B-abliterated
Wizardlm · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
IR
Irbis-7b-v0.1
Irbis · mradermacher
Params7.47B
Context32K
Q4_K_M4.5 GB
QW
WebWorld 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GE
Gemma 4 E4B It Ultra Uncensored Heretic
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
GE
Gemma 4 E4B It The DECKARD Expresso Universe HERETIC UNCENSORED Thinking
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
QW
Gliese Qwen3.5 9B Abliterated Caption
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
JE
Jebadiah 9b v2 1
Jebadiah · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
MI
speechless-mistral-six-in-one-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma-4-E4B-It
Gemma · Hlebek
Params7.52B
Context128K
Q4_K_M5.0 GB
KU
Kukedlc NeuralSynthesis 7B v0.3
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
CO
Q2.5 ColdBrew R1 ForgeWide SAFE
Coldbrew · mradermacher
Params9.5B
Context128K
Q4_K_M5.8 GB
QW
Qwen3.5-9B
Qwen · byteshape
Params8.95B
Context256K
Smallest3.2 GB
TR
Translategemma 4b It
Translategemma · mradermacher
Params3.88B
Context128K
Q4_K_M2.5 GB
GL
Huihui GLM 4.6V Flash Abliterated
GLM · mradermacher
Params9.4B
Context128K
Q4_K_M6.2 GB
TR
Dolphin X1 Trinity Nano
Trinity · dphn
Params6.1B
Context128K
Smallest3.2 GB