Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
MI
Voxtral Mini 3B
Mistral · handy-computer
Params4.7B
Context—
Q4_K_M3.0 GB
GE
gemma-2-9b-it
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
SA
Sao10K L3 8B Stheno v3.1
Sao · RichardErkhov
Params8B
Context—
Q4_K_M0.3 GB
BO
Bonsai-4B
Bonsai · prism-ml
Params4B
Context32K
Smallest0.6 GB
QW
Qwen Image Edit Rapid AIO
Qwen · Phil2Sat
Params7.62B
Context125K
Q4_K_M4.7 GB
GE
gemma 4 E4B it
Gemma · AtomicChat
Params7.52B
Context128K
Q4_K_M5.3 GB
QW
Qwen3 VL 8B Instruct
Qwen · bartowski
Params8.19B
Context256K
Q4_K_M5.0 GB
QW
sakura-7b-qwen2.5-v1.0-iq4xs
Qwen · SakuraLLM
Params7.62B
Context128K
Smallest4.3 GB
GL
Zai org_GLM 4.6V Flash
GLM · lmstudio-community
Params9.4B
Context128K
Q4_K_M6.2 GB
QW
Qwen2.5 7B Instruct
Qwen · lmstudio-community
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
qwen3.5 4B super coder
Qwen · jica98
Params4.33B
Context256K
Smallest2.6 GB
LL
vntl llama3 8b v2
Llama · lmg-anon
Params8B
Context8K
Smallest5.7 GB
LL
Llama Joycaption Beta One Hf Llava
Llama · concedo
Params8B
Context128K
Smallest4.9 GB
MI
avemio digital mistral 7b Sft pretrain 6000 steps merge
Mistral · RichardErkhov
Params7.24B
Context—
Q4_K_M0.1 GB
PH
Phi 3 mini 4k instruct
Phi · microsoft
Params3.82B
Context4K
Smallest2.4 GB
QW
Qwen3 4B RPG Roleplay V2
Qwen · Chun121
Params4B
Context32K
Q4_K_M2.5 GB
SP
Spark X2.6 4B Abliterated
Spark · SC117
Params4.11B
Context1024K
Q4_K_M2.7 GB
GL
Models Zai Org GLM 4.6V Flash
GLM · MaziyarPanahi
Params9.4B
Context128K
Q4_K_M6.2 GB
NE
NeoHorse 1 4B
Neohorse · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
OR
Ornith 1.5 9B Heretic v2
Ornith · Thunder13240
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
codellama_codellama-7b-instruct-hf
Llama · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
MI
MistralRP Noromaid NSFW Mistral 7B
Mistral · Ttimofeyka
Params7.24B
Context32K
Smallest4.1 GB
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
NE
NVIDIA Nemotron Nano 9B v2
Nemotron · bartowski
Params8.89B
Context1024K
Q4_K_M6.5 GB
GE
gemma 7b it
Gemma · google
Params8.54B
Context—
Smallest34.2 GB
TE
HY MT1.5 7B
Tencent · tencent
Params7.5B
Context256K
Q4_K_M4.6 GB
FL
flux klein 9b Biglove Collection
Flux · Granddyser
Params9.08B
Context—
Smallest6.5 GB
QW
Qwen3.5 Text 9B NSFW RP RolePlay
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Huihui Qwen3.8 Flash Next abliterated NVFP4 GPTQ Strata
Qwen · Maximilian228
Params4.95B
Context—
Smallest1.3 GB
GL
Glm 4 9b Chat
GLM · QuantFactory
Params9.4B
Context128K
Q4_K_M6.3 GB