Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
OpenChat 3.5 7B Qwen v2.0
Qwen · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
MI
OpenHermes-2.5-Mistral-7B-x2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mahou-1.2-mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
MA
Magnum Blackout Ataraxy 2 9b i1
Magnum · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
LO
Loki v5.1 i1
Loki · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.8 4B Heretic
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
MI
Ministral 8B Instruct 2410
Mistral · bartowski
Params8B
Context32K
Q4_K_M4.9 GB
JI
JiRackUltra_7b
Jirackultra · mradermacher
Params7.6B
Context128K
Q4_K_M4.7 GB
LL
Llama 3.1 8B Reason Blend 888k i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
ZE
Zen3 Vl
Zen · mradermacher
Params8.2B
Context256K
Q4_K_M5.0 GB
OP
dreamgen_opus-v0-7b
Opus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · roleplaiapp
Params7.62B
Context128K
Q4_K_M4.7 GB
QW
Qwen3.5 9B Brainwaves OneJev1.6
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
LL
Llama-3-Groq-8B-Tool-Use
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
LL
Lexi Llama 3 8B Uncensored
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GR
Granite 4.1 8b
Granite · bartowski
Params8.79B
Context128K
Q4_K_M5.5 GB
LL
Llama 3.1 Minitron 4B Width Base
Llama · ThomasBaruzier
Params4.51B
Context128K
Q4_K_M2.8 GB
JE
JEV 9B
Jev · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
OP
OpenCerebrum 2.0 7B
Opencerebrum · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Asclepius-Llama3-8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen3.5-4B
Qwen · LocalScribe
Params4.21B
Context256K
Q4_K_M2.7 GB
NU
NuMarkdown-8B-Thinking
Numarkdown · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
GE
Gemma 4 E2B It Abliterated
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
QW
Qwen3 8B Base
Qwen · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
NE
NeuralMaths Experiment 7b
Neuralmaths · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
PL
plumb 4b
Plumb · crh225
Params4.2B
Context256K
Q4_K_M2.7 GB
NE
Next 8b
Next · mradermacher
Params8.19B
Context128K
Q4_K_M5.0 GB
IN
Index Translate 9B
Index · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
LL
Cogito v1 Preview Llama 3B
Llama · bartowski
Params3.61B
Context128K
Q4_K_M2.2 GB
GR
Grok Oss Thanatos
Grok · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB