Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3.5-4B
Qwen · weareapexcreators
Params4.33B
Context256K
Q4_K_M2.8 GB
GE
Gemma 4 E2B it LocalAI
Gemma · weareapexcreators
Params7.5B
Context128K
Q4_K_M5.0 GB
MI
open-orca_mistral-7b-openorca
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen Image 2.1 DiT
Qwen · pottokao
Params7.12B
Context—
Q4_K_M4.3 GB
CH
Chandra Ocr 2
Chandra · prithivMLmods
Params4.84B
Context256K
Q4_K_M3.1 GB
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · lmstudio-community
Params7.62B
Context128K
Q4_K_M4.7 GB
FL
flux 3 image klein pup
Flux · unsloth
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
DeepSeek V4 Pro Qwen3.5 9B MTP Public 20260801 HF
Qwen · Jackrong
Params9.2B
Context256K
Q4_K_M5.8 GB
QU
R3GE Quen2.5 7B
Quen · JackFRost0703
Params7.62B
Context128K
Q4_K_M4.7 GB
QW
Qwen3 4B Instruct Awq
Qwen · electricpipelines
Params4B
Context40K
Q4_K_M2.5 GB
LL
Llama-3.1-8B-Instruct
Llama · unsloth
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen2.5 VL 7B Instruct
Qwen · bartowski
Params7.62B
Context125K
Q4_K_M4.7 GB
QW
Qwen3.5-9B
Qwen · weareapexcreators
Params9.2B
Context256K
Q4_K_M5.9 GB
QW
Qwen2.5 7b Ins v3
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
Llama 3.1 8B Stheno v3.4
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
OR
Ornith 1.5 9B
Ornith · abenzerps
Params8.95B
Context256K
Smallest2.8 GB
LL
Anubis Mini 8B v1
Llama · TheDrummer
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen2.5-Omni-7B
Qwen · unsloth
Params7.62B
Context32K
Q4_K_M4.7 GB
OR
Ornith 1.0 9B MTP LocalAI
Ornith · weareapexcreators
Params9.2B
Context256K
Q4_K_M5.8 GB
MI
vllm-d3-mini
Mini · ggml-org
Params4.84B
Context256K
Q4_K_M3.7 GB
QW
Qwen3 VL 8B Instruct Abliterated v2.0
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
QW
Qwen3.5 9B Claude 4.6 HighIQ INSTRUCT HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Mica v0.1 4B
Qwen · sky7350
Params4.84B
Context256K
Q4_K_M3.1 GB
QW
Qwen Image 2.1 Uncensored HF
Qwen · 0xSojalSec
Params7.12B
Context—
Q4_K_M4.6 GB
OP
opendecider small
Opendecider · manjunathshiva
Params4.02B
Context256K
Q4_K_M2.5 GB
QW
LocalAI functioncall qwen2.5 7b v0.5 Q4 K M
Qwen · LocalAI-io
Params7.62B
Context32K
Q4_K_M4.7 GB
JA
Jan-v2-VL-high
Jan · janhq
Params8.2B
Context256K
Q4_K_M5.0 GB
ME
Medgemma-1.5-4B-It
Medgemma · unsloth
Params3.88B
Context128K
Q4_K_M2.5 GB
NA
Nanbeige4.2 3B
Nanbeige · gkraker04
Params4.17B
Context256K
Q4_K_M2.6 GB