Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
ZE
zed industries zeta 2
Zeta · bartowski
Params8.25B
Context32K
Q4_K_M5.2 GB
GGUF
QW
Qwen3 4B Base
Qwen · mradermacher
Params4B
Context32K
Q4_K_M2.5 GB
GGUF
OL
open r1 OlympicCoder 7B
Olympiccoder · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingReasoning
VI
Vinci MLE 8B 1.0
Vinci · mradermacher
Params8.79B
Context128K
Q4_K_M5.3 GB
Tool use
GL
Huihui GLM 4.6V Flash Abliterated
GLM · mradermacher
Params9.4B
Context128K
Q4_K_M6.2 GB
GGUF
QW
MARTHA LXVII.8B QWEN 3.5 3.6 prune 9b 3.6 base
Qwen · ZERO-POINT-AI
Params8.1B
Context256K
Q4_K_M5.1 GB
RoleplayVision
GM
GmshNet 8B v0.1
Gmshnet · hexera-org
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingReasoning
MI
dolphin 2.8 mistral 7b v02
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Models Qwen Qwen2 Math 7B Instruct
Qwen · MaziyarPanahi
Params7.62B
Context4K
Q4_K_M4.7 GB
GGUF
LL
Replete Coder V2 Llama 3.1 8b
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
Coding
WI
wizardlm_wizardcoder-python-7b-v1.0
Wizardlm · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
teknium_hermes-trismegistus-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MI
Chinese Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.5B
Context32K
Q4_K_M4.5 GB
GGUF
LL
Llama3.3 8B Instruct Thinking Claude 4.5 Opus High Reasoning
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
ReasoningRoleplay
MI
voxtral mini 3b 2507
Mistral · cstr
Params4.68B
Context—
Smallest2.7 GB
GGUF
YI
Yi 1.5 9B Chat 16K Abliterated
Yi · mradermacher
Params8.83B
Context16K
Q4_K_M5.3 GB
GGUF
MI
fpham_generate_question_mistral_7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Radha 9B
Qwen · Advisio
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool useVision
FI
Fibo 2.0 Dev and Edit Turbo ComfyUI
Fibo · vantagewithai
Params8.23B
Context—
Q4_K_M6.2 GB
GGUF
NA
calderaai_naberius-7b
Naberius · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Llama 3 Patronus Lynx 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Infinity Instruct 7M Gen Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 9B Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
MI
models--automerger--T3qInex12-7B
Mistral · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CH
chujiezheng Starling LM 7B beta ExPO
Chujiezheng · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 2507 4B Instruct Haiku 4.5 Merged
Qwen · bartowski
Params4B
Context256K
Q4_K_M2.6 GB
GGUF
IR
Irene-RP-v2-7B
Irene · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
AM
Amaretto 8B
Amaretto · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
Tool useVision
LL
Llama-3-8B-Instruct-Gradient-1048k
Llama · crusoeai
Params8B
Context1024K
Q4_K_M4.9 GB
GGUF
MI
uukuguy_speechless-code-mistral-7b-v1.0
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding

Other sizes

Keep browsing