Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Voxtral Mini 3B
Mistral · handy-computer
Params4.7B
Context—
Q4_K_M3.0 GB
GGUF
GE
gemma-2-9b-it
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
SA
Sao10K L3 8B Stheno v3.1
Sao · RichardErkhov
Params8B
Context—
Q4_K_M0.3 GB
GGUF
BO
Bonsai-4B
Bonsai · prism-ml
Params4B
Context32K
Smallest0.6 GB
GGUF
QW
Qwen Image Edit Rapid AIO
Qwen · Phil2Sat
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
GE
gemma 4 E4B it
Gemma · AtomicChat
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
QW
Qwen3 VL 8B Instruct
Qwen · bartowski
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
QW
sakura-7b-qwen2.5-v1.0-iq4xs
Qwen · SakuraLLM
Params7.62B
Context128K
Smallest4.3 GB
GGUF
GL
Zai org_GLM 4.6V Flash
GLM · lmstudio-community
Params9.4B
Context128K
Q4_K_M6.2 GB
Vision
QW
Qwen2.5 7B Instruct
Qwen · lmstudio-community
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
qwen3.5 4B super coder
Qwen · jica98
Params4.33B
Context256K
Smallest2.6 GB
CodingReasoningTool use
LL
vntl llama3 8b v2
Llama · lmg-anon
Params8B
Context8K
Smallest5.7 GB
GGUF
LL
Llama Joycaption Beta One Hf Llava
Llama · concedo
Params8B
Context128K
Smallest4.9 GB
Vision
MI
avemio digital mistral 7b Sft pretrain 6000 steps merge
Mistral · RichardErkhov
Params7.24B
Context—
Q4_K_M0.1 GB
GGUF
PH
Phi 3 mini 4k instruct
Phi · microsoft
Params3.82B
Context4K
Smallest2.4 GB
Coding
QW
Qwen3 4B RPG Roleplay V2
Qwen · Chun121
Params4B
Context32K
Q4_K_M2.5 GB
Roleplay
SP
Spark X2.6 4B Abliterated
Spark · SC117
Params4.11B
Context1024K
Q4_K_M2.7 GB
ReasoningTool use
GL
Models Zai Org GLM 4.6V Flash
GLM · MaziyarPanahi
Params9.4B
Context128K
Q4_K_M6.2 GB
GGUF
NE
NeoHorse 1 4B
Neohorse · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
OR
Ornith 1.5 9B Heretic v2
Ornith · Thunder13240
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
LL
codellama_codellama-7b-instruct-hf
Llama · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
MistralRP Noromaid NSFW Mistral 7B
Mistral · Ttimofeyka
Params7.24B
Context32K
Smallest4.1 GB
GGUF
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
NE
NVIDIA Nemotron Nano 9B v2
Nemotron · bartowski
Params8.89B
Context1024K
Q4_K_M6.5 GB
GGUF
GE
gemma 7b it
Gemma · google
Params8.54B
Context—
Smallest34.2 GB
GGUF
TE
HY MT1.5 7B
Tencent · tencent
Params7.5B
Context256K
Q4_K_M4.6 GB
GGUF
FL
flux klein 9b Biglove Collection
Flux · Granddyser
Params9.08B
Context—
Smallest6.5 GB
GGUF
QW
Qwen3.5 Text 9B NSFW RP RolePlay
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Roleplay
QW
Huihui Qwen3.8 Flash Next abliterated NVFP4 GPTQ Strata
Qwen · Maximilian228
Params4.95B
Context—
Smallest1.3 GB
GGUF
GL
Glm 4 9b Chat
GLM · QuantFactory
Params9.4B
Context128K
Q4_K_M6.3 GB
GGUF

Other sizes

Keep browsing