Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
ZE
zed industries zeta 2
Zeta · bartowski
Params8.25B
Context32K
Q4_K_M5.2 GB
QW
Qwen3 4B Base
Qwen · mradermacher
Params4B
Context32K
Q4_K_M2.5 GB
OL
open r1 OlympicCoder 7B
Olympiccoder · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
VI
Vinci MLE 8B 1.0
Vinci · mradermacher
Params8.79B
Context128K
Q4_K_M5.3 GB
GL
Huihui GLM 4.6V Flash Abliterated
GLM · mradermacher
Params9.4B
Context128K
Q4_K_M6.2 GB
QW
MARTHA LXVII.8B QWEN 3.5 3.6 prune 9b 3.6 base
Qwen · ZERO-POINT-AI
Params8.1B
Context256K
Q4_K_M5.1 GB
GM
GmshNet 8B v0.1
Gmshnet · hexera-org
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
dolphin 2.8 mistral 7b v02
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Models Qwen Qwen2 Math 7B Instruct
Qwen · MaziyarPanahi
Params7.62B
Context4K
Q4_K_M4.7 GB
LL
Replete Coder V2 Llama 3.1 8b
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
WI
wizardlm_wizardcoder-python-7b-v1.0
Wizardlm · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
MI
teknium_hermes-trismegistus-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Chinese Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.5B
Context32K
Q4_K_M4.5 GB
LL
Llama3.3 8B Instruct Thinking Claude 4.5 Opus High Reasoning
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
MI
voxtral mini 3b 2507
Mistral · cstr
Params4.68B
Context—
Smallest2.7 GB
YI
Yi 1.5 9B Chat 16K Abliterated
Yi · mradermacher
Params8.83B
Context16K
Q4_K_M5.3 GB
MI
fpham_generate_question_mistral_7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Radha 9B
Qwen · Advisio
Params9.2B
Context256K
Q4_K_M5.8 GB
FI
Fibo 2.0 Dev and Edit Turbo ComfyUI
Fibo · vantagewithai
Params8.23B
Context—
Q4_K_M6.2 GB
NA
calderaai_naberius-7b
Naberius · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3 Patronus Lynx 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
MI
Infinity Instruct 7M Gen Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.5 9B Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
MI
models--automerger--T3qInex12-7B
Mistral · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
CH
chujiezheng Starling LM 7B beta ExPO
Chujiezheng · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
QW
Qwen3 2507 4B Instruct Haiku 4.5 Merged
Qwen · bartowski
Params4B
Context256K
Q4_K_M2.6 GB
IR
Irene-RP-v2-7B
Irene · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
AM
Amaretto 8B
Amaretto · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
LL
Llama-3-8B-Instruct-Gradient-1048k
Llama · crusoeai
Params8B
Context1024K
Q4_K_M4.9 GB
MI
uukuguy_speechless-code-mistral-7b-v1.0
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB