Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
GE
Gemma 4 E4B It Erotica
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
SC
ajibawa-2023_scarlett-7b
Scarlett · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
DE
Llama 3.1 TAIDE R1 8B Chat
DeepSeek · voidful
Params8.52B
Context128K
Smallest17.1 GB
LL
Llama 3.1 Storm 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen3.5 9B Artemis
Qwen · turintech
Params8.95B
Context256K
Q4_K_M5.7 GB
LL
LLaDA 1.5
Llada · mradermacher
Params8.02B
Context4K
Q4_K_M4.9 GB
GE
gemma 3n E2B it
Gemma · ggml-org
Params4.46B
Context32K
Smallest4.8 GB
JA
Javi0.3 4B
Javi · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
AI
airoboros l2 7B 2.2.1
Airoboros · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GR
Granite 3.2 8b Instruct
Granite · lmstudio-community
Params8.17B
Context128K
Q4_K_M4.9 GB
QW
Qwen3 Psychological Reasoning 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
AT
Athena-CodeZephyr-7B-v0.2
Athena · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
deepseek coder 6.7B kexer
DeepSeek · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
MI
Hermes-2-Pro-Mistral-7B-infinity
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwimi 4B
Qwimi · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
TI
Tifa DeepsexV2 7b MGRPO GGUF Q8
Tifa · ValueFX9507
Params7.6B
Context32K
Smallest8.1 GB
GR
Granite Switch 4.1 8b
Granite · ibm-granite
Params8.97B
Context128K
Q4_K_M6.4 GB
HE
HERETICSEEK 7B Ditill
Hereticseek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
PI
maywell_pivot-0.1-early
Pivot · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
TH
TherapyLlama 8B v1 i1
Therapyllama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
MI
MiniAI Quata2 4b
Miniai · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
ST
starcoder2 7b
StarCoder · QuantFactory
Params7.2B
Context16K
Q4_K_M4.5 GB
PP
pplx decider flash v1.2
Pplx · Jakevin
Params4.84B
Context256K
Q4_K_M3.1 GB
TE
Teutonic I Reliquary RL
Teutonic · bartowski
Params8.95B
Context256K
Q4_K_M5.8 GB
QW
Meissa Qwen2.5 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
LL
Infinity Instruct 7M Gen Llama3_1 8B
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GE
Gemma 4 E2B It Ultra Uncensored Heretic
Gemma · princepppp
Params4.65B
Context128K
Q4_K_M3.4 GB
ST
L3-8B-Stheno-v3.3-32K
Stheno · Lewdiculous
Params8B
Context32K
Q4_K_M4.9 GB
GR
Granite 8b Code Instruct 4k
Granite · mradermacher
Params8.05B
Context4K
Q4_K_M4.9 GB