Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5-4B
Qwen · weareapexcreators
Params4.33B
Context256K
Q4_K_M2.8 GB
Vision
GE
Gemma 4 E2B it LocalAI
Gemma · weareapexcreators
Params7.5B
Context128K
Q4_K_M5.0 GB
Vision
MI
open-orca_mistral-7b-openorca
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen Image 2.1 DiT
Qwen · pottokao
Params7.12B
Context—
Q4_K_M4.3 GB
GGUF
CH
Chandra Ocr 2
Chandra · prithivMLmods
Params4.84B
Context256K
Q4_K_M3.1 GB
ReasoningTool useVision
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · lmstudio-community
Params7.62B
Context128K
Q4_K_M4.7 GB
Reasoning
FL
flux 3 image klein pup
Flux · unsloth
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
QW
DeepSeek V4 Pro Qwen3.5 9B MTP Public 20260801 HF
Qwen · Jackrong
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool useVision
QU
R3GE Quen2.5 7B
Quen · JackFRost0703
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
QW
Qwen3 4B Instruct Awq
Qwen · electricpipelines
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
LL
Llama-3.1-8B-Instruct
Llama · unsloth
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 VL 7B Instruct
Qwen · bartowski
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
QW
Qwen3.5-9B
Qwen · weareapexcreators
Params9.2B
Context256K
Q4_K_M5.9 GB
Vision
QW
Qwen2.5 7b Ins v3
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Llama 3.1 8B Stheno v3.4
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
OR
Ornith 1.5 9B
Ornith · abenzerps
Params8.95B
Context256K
Smallest2.8 GB
ReasoningTool use
LL
Anubis Mini 8B v1
Llama · TheDrummer
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5-Omni-7B
Qwen · unsloth
Params7.62B
Context32K
Q4_K_M4.7 GB
Vision
OR
Ornith 1.0 9B MTP LocalAI
Ornith · weareapexcreators
Params9.2B
Context256K
Q4_K_M5.8 GB
GGUF
MI
vllm-d3-mini
Mini · ggml-org
Params4.84B
Context256K
Q4_K_M3.7 GB
Vision
QW
Qwen3 VL 8B Instruct Abliterated v2.0
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Tool useVision
QW
Qwen3.5 9B Claude 4.6 HighIQ INSTRUCT HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningRoleplayTool use
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Mica v0.1 4B
Qwen · sky7350
Params4.84B
Context256K
Q4_K_M3.1 GB
GGUF
QW
Qwen Image 2.1 Uncensored HF
Qwen · 0xSojalSec
Params7.12B
Context—
Q4_K_M4.6 GB
GGUF
OP
opendecider small
Opendecider · manjunathshiva
Params4.02B
Context256K
Q4_K_M2.5 GB
Tool use
QW
LocalAI functioncall qwen2.5 7b v0.5 Q4 K M
Qwen · LocalAI-io
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
JA
Jan-v2-VL-high
Jan · janhq
Params8.2B
Context256K
Q4_K_M5.0 GB
Vision
ME
Medgemma-1.5-4B-It
Medgemma · unsloth
Params3.88B
Context128K
Q4_K_M2.5 GB
ReasoningVision
NA
Nanbeige4.2 3B
Nanbeige · gkraker04
Params4.17B
Context256K
Q4_K_M2.6 GB
GGUF

Other sizes

Keep browsing