Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5 9B Claude 4.6 Opus Deckard V4.2 Uncensored Heretic Thinking
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningRoleplay
QW
Qwen3.8 4B Distill Heretic
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool useVision
EX
Experiment26 7B
Experiment · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
CoPaw Flash 9B
Qwen · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Vision
PA
neuralnovel_panda-7b-v0.1
Panda · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ZE
zephyr 7b hh dpo
Zephyr · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma-2-9B-It-SPPO-Iter3
Gemma · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
DE
DeciLM 7B instruct
Decilm · Deci
Params7.11B
Context8K
Smallest7.6 GB
GGUF
YI
Yi-1.5-9B-32K
Yi · mradermacher
Params8.83B
Context32K
Q4_K_M5.3 GB
GGUF
MI
Ministral 3 8B Reasoning 2512 heretic
Mistral · coder3101
Params8.5B
Context256K
Q4_K_M5.2 GB
CodingReasoningVision
JA
Jais Adapted 7b Chat
Jais · mradermacher
Params7B
Context4K
Q4_K_M4.3 GB
Coding
KU
Kuvera 8B v0.1.0
Kuvera · bartowski
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
QW
Qwen_Qwen3 4B Thinking 2507
Qwen · Alcoft
Params4B
Context256K
Q4_K_M2.4 GB
Reasoning
QW
Qwen2.5 7B Instruct 1M Heretic
Qwen · mradermacher
Params7.62B
Context986K
Q4_K_M4.7 GB
Roleplay
NE
NVIDIA Nemotron Nano 9B v2 Japanese
Nemotron · mmnga-o
Params8.89B
Context1024K
Q4_K_M6.5 GB
GGUF
TE
TerraSense
Terrasense · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Tool useVision
LL
Fireball Alpaca Llama3.1 8B Philos i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
PH
microsoft Phi 3 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
nsfwcaption qwen3 vl 8b v3
Qwen · GitMylo
Params8.2B
Context256K
Q4_K_M5.0 GB
Vision
WI
WizardMath 7B V1.0
Wizardmath · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.5 4B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningRoleplay
GE
gemma 3n E4B it
Gemma · ggml-org
Params6.9B
Context32K
Smallest7.4 GB
GGUF
MY
MythoLogic-Mini-7b
Mythologic · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
NU
NuminaMath-7B-TIR
Numinamath · legraphista
Params6.91B
Context4K
Smallest1.7 GB
GGUF
TE
Ternary Bonsai 4b Abliterated v2
Ternary · Rootkit7
Params4B
Context32K
Q4_K_M2.5 GB
GGUF
GE
gemma 4 E2B it heretic
Gemma · saidutta69
Params4.63B
Context128K
Q4_K_M3.4 GB
ReasoningTool useVision
QW
Qwen3 4B Instruct 2507 Abliterated
Qwen · mradermacher
Params4.02B
Context256K
Q4_K_M2.5 GB
Tool use
QW
Qwen3 Reranker 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
Embedding

Other sizes

Keep browsing