Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Qwen2 Audio 7B Instruct
Qwen · mradermacher
Params7.75B
Context8K
Q4_K_M4.8 GB
PH
Phi 3.1 mini 4k instruct
Phi · bartowski
Params3.82B
Context4K
Q4_K_M2.4 GB
IN
Intern Decision 4B
InternLM · bombdefuser-124
Params4.21B
Context256K
Smallest4.5 GB
ID
ideogram 4
Ideogram · rectangleworm
Params8.19B
Context256K
Q4_K_M5.0 GB
OL
OLMoE 1B 7B 0924 Instruct
OLMo · bartowski
Params6.9B
Context4K
Q4_K_M4.2 GB
FA
Fara1.5 4B
Fara · bartowski
Params4.21B
Context256K
Q4_K_M2.9 GB
QW
Qwen3.8 Flash Next MTP
Qwen · drluoto
Params3.9B
Context256K
Q4_K_M2.8 GB
QW
Qwen3 8B Heretic
Qwen · saidutta69
Params8.19B
Context40K
Q4_K_M5.0 GB
CO
Yi Coder 9B Chat
Coder · RichardErkhov
Params8.83B
Context128K
Q4_K_M5.3 GB
HO
Holo 3.1 9B
Holo · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
MO
Moss Image SFT
Moss · axiomofmind
Params5.08B
Context—
Smallest3.5 GB
MI
teknium_openhermes-2-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
PP
pplx decider flash gguf v1.2
Pplx · perplexity-ai
Params4.84B
Context256K
Smallest5.2 GB
LL
SageV2 8B LongContext NSFW RP
Llama · sinhal
Params8.03B
Context128K
Smallest3.2 GB
DE
DeepSeek R1 Distill Qwen 7B Uncensored
DeepSeek · mradermacher
Params7.61B
Context128K
Q4_K_M4.7 GB
SP
Spark X2.5 4B Abliterated
Spark · SC117
Params4.11B
Context1024K
Smallest2.4 GB
QW
Qwen3.5 9B Claude 4.6 HighIQ THINKING HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
LA
latxa-7b-v1.2
Latxa · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen3.5 9B
Qwen · SoAIHQ
Params9.2B
Context256K
Q4_K_M5.8 GB
LI
Ling 3.0 tiny
Ling · NANI-Nithin
Params7.89B
Context128K
Q4_K_M4.8 GB
HU
HunyuanVideo 1.5 I2V 720p
Hunyuanvideo · jayn7
Params8.33B
Context—
Q4_K_M5.1 GB
QW
Qwen2.5 VL 7B Abliterated Caption It
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
QW
Qwen3.5 4B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
LL
Llama 3.1 8B Lexi Uncensored
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
QW
Andy 4.2
Qwen · Mindcraft-CE
Params8.95B
Context256K
Q4_K_M5.6 GB
MI
ehartford_dolphin-2.2.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 4B Abliterated
Qwen · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
QW
MiMo-V2.6-Distill-Qwen-9B
Qwen · ggml-org
Params8.95B
Context256K
Smallest9.5 GB
LL
Llama3.1-8B-Chinese-Chat
Llama · shenzhi-wang
Params8B
Context128K
Q4_K_M4.9 GB
DE
DeepSeek R1 0528 Qwen3 8B
DeepSeek · bartowski
Params8.19B
Context128K
Q4_K_M5.0 GB