Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2 Audio 7B Instruct
Qwen · mradermacher
Params7.75B
Context8K
Q4_K_M4.8 GB
Vision
PH
Phi 3.1 mini 4k instruct
Phi · bartowski
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
IN
Intern Decision 4B
InternLM · bombdefuser-124
Params4.21B
Context256K
Smallest4.5 GB
Vision
ID
ideogram 4
Ideogram · rectangleworm
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
OL
OLMoE 1B 7B 0924 Instruct
OLMo · bartowski
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
FA
Fara1.5 4B
Fara · bartowski
Params4.21B
Context256K
Q4_K_M2.9 GB
Vision
QW
Qwen3.8 Flash Next MTP
Qwen · drluoto
Params3.9B
Context256K
Q4_K_M2.8 GB
GGUF
QW
Qwen3 8B Heretic
Qwen · saidutta69
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
CO
Yi Coder 9B Chat
Coder · RichardErkhov
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
HO
Holo 3.1 9B
Holo · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
MO
Moss Image SFT
Moss · axiomofmind
Params5.08B
Context—
Smallest3.5 GB
GGUF
MI
teknium_openhermes-2-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
PP
pplx decider flash gguf v1.2
Pplx · perplexity-ai
Params4.84B
Context256K
Smallest5.2 GB
Vision
LL
SageV2 8B LongContext NSFW RP
Llama · sinhal
Params8.03B
Context128K
Smallest3.2 GB
Roleplay
DE
DeepSeek R1 Distill Qwen 7B Uncensored
DeepSeek · mradermacher
Params7.61B
Context128K
Q4_K_M4.7 GB
Reasoning
SP
Spark X2.5 4B Abliterated
Spark · SC117
Params4.11B
Context1024K
Smallest2.4 GB
GGUF
QW
Qwen3.5 9B Claude 4.6 HighIQ THINKING HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningRoleplayTool use
LA
latxa-7b-v1.2
Latxa · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.5 9B
Qwen · SoAIHQ
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool useVision
LI
Ling 3.0 tiny
Ling · NANI-Nithin
Params7.89B
Context128K
Q4_K_M4.8 GB
Reasoning
HU
HunyuanVideo 1.5 I2V 720p
Hunyuanvideo · jayn7
Params8.33B
Context—
Q4_K_M5.1 GB
GGUF
QW
Qwen2.5 VL 7B Abliterated Caption It
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
QW
Qwen3.5 4B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningRoleplayVision
LL
Llama 3.1 8B Lexi Uncensored
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Andy 4.2
Qwen · Mindcraft-CE
Params8.95B
Context256K
Q4_K_M5.6 GB
Tool useVision
MI
ehartford_dolphin-2.2.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 4B Abliterated
Qwen · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
QW
MiMo-V2.6-Distill-Qwen-9B
Qwen · ggml-org
Params8.95B
Context256K
Smallest9.5 GB
Vision
LL
Llama3.1-8B-Chinese-Chat
Llama · shenzhi-wang
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
DE
DeepSeek R1 0528 Qwen3 8B
DeepSeek · bartowski
Params8.19B
Context128K
Q4_K_M5.0 GB
Reasoning

Other sizes

Keep browsing