Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
OL
Olmo-3-7B-Instruct
OLMo · unsloth
Params7.3B
Context64K
Q4_K_M4.5 GB
GGUF
QW
Qwen3 4B Instruct 2507 Heretic
Qwen · bartowski
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
NA
Nanbeige4.2 3B
Nanbeige · owao
Params4.17B
Context256K
Q4_K_M2.6 GB
GGUF
QW
Qwen3.5-9B
Qwen · jc-builds
Params8.95B
Context256K
Q4_K_M5.7 GB
Reasoning
IN
index echo 9b
Index · cstr
Params8.95B
Context—
Smallest1.3 GB
GGUF
GU
GUI Owl 1.5 8B Instruct
Gui · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
Vision
PY
Pythia 6.9b
Pythia · mradermacher
Params6.86B
Context2K
Q4_K_M4.4 GB
GGUF
HE
HERO 9B
Hero · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
FI
Finch 7b
Finch · latestissue
Params7.64B
Context1024K
Smallest3.0 GB
GGUF
QW
Qwen Image 2.1 viggle turbo v0.2.1 6step
Qwen · Abiray
Params7.12B
Context—
Q4_K_M4.2 GB
GGUF
QW
Qwen3 VL 8B NSFW Caption V4.5 Heretic
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
AE
AesCode 8B
Aescode · bartowski
Params8.19B
Context256K
Q4_K_M5.2 GB
CodingVision
LL
MiniCPM Llama3 V 2 5
Llama · openbmb
Params8.2B
Context8K
Q4_K_M4.9 GB
Vision
MO
MOSS Image
Moss · molbal
Params5.08B
Context—
Q4_K_M3.6 GB
GGUF
DE
DeepSeek R1 Distill Llama 8B
DeepSeek · lmstudio-community
Params8B
Context128K
Q4_K_M4.9 GB
Reasoning
RO
RoLlama2-7b-Base
Rollama · legraphista
Params6.74B
Context4K
Smallest1.5 GB
GGUF
ME
Medgemma 4b It
Medgemma · bartowski
Params3.88B
Context128K
Q4_K_M2.5 GB
ReasoningVision
FA
FastWan2.2 TI2V 5B FullAttn
Fastwan · Green-Sky
Params5B
Context—
Smallest4.2 GB
GGUF
LL
FILM6912 Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8B
Context—
Q4_K_M0.0 GB
GGUF
GR
Granite 3.3 8b Instruct
Granite · ibm-granite
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama Artemis
Llama · mradermacher
Params8.2B
Context128K
Q4_K_M5.0 GB
GGUF
MI
MiniCPM V 4
MiniCPM · openbmb
Params3.6B
Context32K
Q4_K_M2.2 GB
CodingVision
DE
Qwen3.5 9B Distill Deepseek V4.1 Flash 20261004
DeepSeek · kevinzhou21
Params9.2B
Context256K
Q4_K_M5.8 GB
Vision
FL
Flux2 Klein 9b Uncensored Text Encoder
Flux · WANG-CC
Params8.19B
Context40K
Smallest8.7 GB
Coding
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · lumen-models
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen3 4B Instruct 2507 Polaris Alpha Distill Heretic Abliterated
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
IM
Z-Image-Engineer-V6-SMART-DoRA-StyleBlendV9-25V17-75
Image · BennyDaBall
Params4B
Context40K
Q4_K_M2.5 GB
Coding
MI
Ming Image GGUFs
Ming · realrebelai
Params6.15B
Context—
Q4_K_M5.9 GB
GGUF
RW
Rwkv 6 World 7b
Rwkv · latestissue
Params7.64B
Context1024K
Smallest3.0 GB
GGUF
KE
Kev 9B
Kev · ggml-org
Params9B
Context256K
Q4_K_M6.4 GB
GGUF

Other sizes

Keep browsing