Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
ST
stable diffusion 3.5 large
Stable · city96
Params8.15B
Context—
Smallest4.8 GB
GGUF
FA
Fara1.5 9B
Fara · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Vision
QW
Gliese Qwen3.5 9B Abliterated Caption
Qwen · prithivMLmods
Params8.95B
Context256K
Smallest9.5 GB
ReasoningVision
GE
Gemma 4 E2B it Artemis
Gemma · turintech
Params4.63B
Context128K
Smallest0.1 GB
ReasoningTool useVision
NE
Huihui NeoHorse 1 4B Abliterated MTP
Neohorse · huihui-ai
Params4.33B
Context256K
Smallest2.3 GB
ReasoningTool useVision
QW
Qwen3.8 9B Heretic Uncensored
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
GR
Granite 4.0 H Tiny
Granite · ibm-granite
Params6.94B
Context1024K
Q4_K_M4.2 GB
GGUF
PH
Phi 3 mini 4k instruct
Phi · bartowski
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
SP
speechless zephyr code functionary 7b
Speechless · uukuguy
Params7.24B
Context32K
Q4_K_M4.4 GB
CodingTool use
QW
Qwen3 4B Thinking 2507
Qwen · lmstudio-community
Params4B
Context256K
Q4_K_M2.5 GB
Reasoning
QW
Qwen3Vl 8b Thinking
Qwen · Qwen
Params8.19B
Context256K
Q4_K_M5.0 GB
ReasoningVision
MU
music flamingo
Music · henry1477
Params7.6B
Context32K
Smallest3.0 GB
ReasoningVision
ER
ERNIE Image Turbo
Ernie · unsloth
Params8.03B
Context—
Q4_K_M5.0 GB
GGUF
QW
Qwen2.5 7B Instruct Uncensored
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
GGUF
GL
Glm-Z1-9B-0414
GLM · unsloth
Params9.4B
Context32K
Q4_K_M6.2 GB
GGUF
QW
Qwen2.5.1 Coder 7B Instruct
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
NE
Nvidia-Nemotron-3-Nano-4B
Nemotron · unsloth
Params3.97B
Context1024K
Q4_K_M2.9 GB
GGUF
AP
APUS Reflex 4B
Apus · apus-ailab
Params4.2B
Context256K
Q4_K_M2.7 GB
GGUF
DE
DeepSeek R1 0528 Qwen3 8B Abliterated
DeepSeek · noteventhrice
Params8.19B
Context128K
Q4_K_M5.0 GB
ReasoningTool use
QW
Qwen3-Vl-8B-Thinking
Qwen · unsloth
Params8.19B
Context256K
Q4_K_M5.0 GB
ReasoningVision
GE
Gemma 4 E2B It Heretic v1
Gemma · mradermacher
Params4.63B
Context128K
Q4_K_M3.4 GB
Vision
ON
OneJev 9B
Onejev · bartowski
Params8.95B
Context256K
Q4_K_M5.8 GB
Vision
HU
Huihui4 8B A4B
Huihui · huihui-ai
Params8.1B
Context256K
Smallest5.4 GB
Vision
QW
Qwen2.5 Coder 7B Instruct
Qwen · unsloth
Params7.62B
Context128K
Q4_K_M4.7 GB
Coding
QW
Josiefied Qwen3 8B Abliterated v1
Qwen · bartowski
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
PH
Phi 3 mini 128k instruct GGUF Imatrix smashed
Phi · PrunaAI
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
AI
Airoboros L2 7B 2.1
Airoboros · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
CY
CyberSecQwen 4B
Cybersecqwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
LL
codellama_codellama-7b-hf
Llama · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
QW
Qwen3.5 9B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING MAX NEOCODE Imatrix
Qwen · DavidAU
Params8.95B
Context256K
Q4_K_M6.8 GB
ReasoningRoleplayVision

Other sizes

Keep browsing