Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3.5 9B Claude 4.6 Opus Deckard V4.2 Uncensored Heretic Thinking
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen3.8 4B Distill Heretic
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
EX
Experiment26 7B
Experiment · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
CoPaw Flash 9B
Qwen · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
PA
neuralnovel_panda-7b-v0.1
Panda · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
ZE
zephyr 7b hh dpo
Zephyr · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma-2-9B-It-SPPO-Iter3
Gemma · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
DE
DeciLM 7B instruct
Decilm · Deci
Params7.11B
Context8K
Smallest7.6 GB
YI
Yi-1.5-9B-32K
Yi · mradermacher
Params8.83B
Context32K
Q4_K_M5.3 GB
MI
Ministral 3 8B Reasoning 2512 heretic
Mistral · coder3101
Params8.5B
Context256K
Q4_K_M5.2 GB
JA
Jais Adapted 7b Chat
Jais · mradermacher
Params7B
Context4K
Q4_K_M4.3 GB
KU
Kuvera 8B v0.1.0
Kuvera · bartowski
Params8.19B
Context40K
Q4_K_M5.0 GB
QW
Qwen_Qwen3 4B Thinking 2507
Qwen · Alcoft
Params4B
Context256K
Q4_K_M2.4 GB
QW
Qwen2.5 7B Instruct 1M Heretic
Qwen · mradermacher
Params7.62B
Context986K
Q4_K_M4.7 GB
NE
NVIDIA Nemotron Nano 9B v2 Japanese
Nemotron · mmnga-o
Params8.89B
Context1024K
Q4_K_M6.5 GB
TE
TerraSense
Terrasense · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
LL
Fireball Alpaca Llama3.1 8B Philos i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
PH
microsoft Phi 3 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
MI
Mistral 7B Instruct v0.2
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
nsfwcaption qwen3 vl 8b v3
Qwen · GitMylo
Params8.2B
Context256K
Q4_K_M5.0 GB
WI
WizardMath 7B V1.0
Wizardmath · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
QW
Qwen3.5 4B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
GE
gemma 3n E4B it
Gemma · ggml-org
Params6.9B
Context32K
Smallest7.4 GB
MY
MythoLogic-Mini-7b
Mythologic · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
NU
NuminaMath-7B-TIR
Numinamath · legraphista
Params6.91B
Context4K
Smallest1.7 GB
TE
Ternary Bonsai 4b Abliterated v2
Ternary · Rootkit7
Params4B
Context32K
Q4_K_M2.5 GB
GE
gemma 4 E2B it heretic
Gemma · saidutta69
Params4.63B
Context128K
Q4_K_M3.4 GB
QW
Qwen3 4B Instruct 2507 Abliterated
Qwen · mradermacher
Params4.02B
Context256K
Q4_K_M2.5 GB
QW
Qwen3 Reranker 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB