Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
GR
Granite 3.3 8b Instruct
Granite · mrutkows
Params8.17B
Context128K
Q4_K_M4.9 GB
CO
ComfyUI Starter Packs
Comfyui · MaxedOut
Params4.76B
Context—
Q4_K_M9.7 GB
QW
qwen3.5 9b null space abliterated
Qwen · jwest33
Params8.95B
Context256K
Smallest7.4 GB
IN
InSight Doc 8B
Insight · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
QW
Qwen3 4B Instruct 2507
Qwen · a-ivanovitch
Params4B
Context256K
Q4_K_M2.5 GB
PE
Peach 2.0 9B 8k Roleplay
Peach · mradermacher
Params8.83B
Context16K
Q4_K_M5.3 GB
LL
Llama 3.1 8B MultiReflection Instruct i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GL
LongWriter Glm4 9b
GLM · QuantFactory
Params9.4B
Context1024K
Q4_K_M6.3 GB
IM
POCKET Zimage CPU
Image · FINAL-Bench
Params6.15B
Context—
Smallest3.5 GB
MI
dolphin-2.9.3-mistral-7B-32k
Mistral · bartowski
Params7.25B
Context32K
Q4_K_M4.4 GB
ST
Starling LM 7B alpha ExPO
Starling · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GE
Amoral Gemma3 4B
Gemma · bartowski
Params3.88B
Context128K
Q4_K_M2.5 GB
KR
Kronumos
Kronumos · mradermacher
Params7.6B
Context32K
Q4_K_M4.7 GB
BA
bagel 8b v1.0 i1
Bagel · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen3.8 9B
Qwen · hakergeniusz
Params9.2B
Context256K
Q4_K_M5.8 GB
QW
Qwen 3 VL 8B Instruct Heretic
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
GE
Gemma 2 9b It
Gemma · duyntnet
Params9.24B
Context8K
Q4_K_M5.8 GB
LL
Llama2 7B Hindi finetuned
Llama · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB
MO
models--automerger--Inex12T3qm7xp-7B
MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
TE
Tema_Q R2.0 Code
Tema · mradermacher
Params8.5B
Context8K
Q4_K_M5.3 GB
LL
Hermes-2-Theta-Llama-3-8B-32k
Llama · mradermacher
Params8B
Context32K
Q4_K_M4.9 GB
GE
Baihua Gemma4 E4B
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
SA
SauerkrautLM 7b LaserChat
Sauerkrautlm · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
RO
L3.2 Rogue Creative Instruct 7B
Rogue · DavidAU
Params7.14B
Context128K
Q4_K_M4.4 GB
LL
deepthought 8b llama v0.01 alpha
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen3.5 9B The Bradbury F451 Pro Writer Uncensored Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
MI
arc53_docsgpt-7b-mistral
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
CH
ChatLaw2E_plain_7B
Chatlaw · mradermacher
Params7.74B
Context32K
Q4_K_M4.7 GB
LL
Llama-3.3-8B-Instruct
Llama · Mungert
Params8B
Context8K
Q4_K_M5.3 GB
DO
Dolphin3 Cyber 8B
Dolphin · ScottzillaSystems
Params8B
Context128K
Q4_K_M4.9 GB