Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
TE
Teknofest 2025 Turkish Edu v2
Teknofest · mradermacher
Params8.2B
Context40K
Q4_K_M5.0 GB
GGUF
QW
GrayLine Qwen3 8B
Qwen · bartowski
Params8.19B
Context40K
Q4_K_M5.0 GB
Reasoning
DO
Dolphin3 Cyber 8B
Dolphin · meowmeow123ok
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GE
Gemma 4 E2B
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
Vision
GR
Granite 8b Code Base
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
Coding
QW
Qwen2.5 7B Instruct Uncensored
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
PH
Borea Phi 3.5 Mini Instruct Jp
Phi · mmnga
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
OL
OLMo-1.7-7B-hf
OLMo · mradermacher
Params6.89B
Context4K
Q4_K_M4.2 GB
GGUF
LL
Llama 3.1 SuperNova Lite
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MI
bitext Mistral 7B Customer Support
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IN
InternVL3_5 8B
Internvl · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
CodingVision
NO
Novel Humanize 4b
Novel · mradermacher
Params4.02B
Context40K
Q4_K_M2.5 GB
ReasoningTool use
TA
UI TARS 7B DPO
Tars · lmstudio-community
Params7.62B
Context32K
Q4_K_M4.7 GB
Vision
TO
ToolSpace 8B
Toolspace · mradermacher
Params8.19B
Context64K
Q4_K_M5.0 GB
ReasoningTool use
KO
Konstanta V4 Alpha 7B
Konstanta · mradermacher
Params7.24B
Context8K
Q4_K_M4.5 GB
Roleplay
MI
mistralai_Ministral 3 8B Reasoning 2512
Mistral · lmstudio-community
Params8.49B
Context256K
Q4_K_M5.2 GB
ReasoningVision
MI
rana shahroz mistral 7b roleplay alpaca merge
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
VU
VulnLLM R 7B
Vulnllm · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingReasoning
QW
Qwen3.5 9B OmniCoder Claude Polaris
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingRoleplay
IN
InverseCoder-DS-6.7B
Inversecoder · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
JA
Jan-Nano
Jan · unsloth
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
GE
gemma 4 E2B it qat q4 0 heretic
Gemma · FadedRedStar
Params4.63B
Context128K
Smallest0.1 GB
CodingVision
GE
Gemma 4 E2B It Uncensored MAX
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
Reasoning
QW
Qwopus3.5 4B v3
Qwen · Jackrong
Params4.2B
Context256K
Q4_K_M2.7 GB
ReasoningVision
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
ST
StructLM-7B
Structlm · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
LL
LongWriter Llama3.1 8b
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
FL
Flux 3 Image Klein Pup
Flux · ND911
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
QW
Qwen3.8 Flash Next MTP Q8 0
Qwen · agentionai
Params3.88B
Context256K
Smallest4.1 GB
GGUF
MI
Clinical BR Mistral 7B v0.2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other sizes

Keep browsing