Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 Math 7B Instruct
Qwen · lmstudio-community
Params7.62B
Context4K
Q4_K_M4.7 GB
GGUF
NE
NeoHorse-Jev-4B Q4_K_M unified decision + vision
Neohorse · TokenRhythm
Params4.2B
Context256K
Q4_K_M3.4 GB
Vision
UC
UCLA AGI zephyr 7b sft full SPIN iter3
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5-Vl-7B-Instruct
Qwen · mnsims1024
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
MI
Nexus IKM Mistral Instruct v0.2 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
YO
Youtu-VL-4B-Instruct
Youtu · Mungert
Params4.9B
Context32K
Q4_K_M3.1 GB
Vision
MA
Marco o1
Marco · bartowski
Params7.6B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct
Qwen · DuoNeural
Params7.6B
Context32K
Q4_K_M4.7 GB
Coding
DE
realtreetune deepseekmath 7b sft GSM8K
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
GE
Gemma 3 4b It Uncensored
Gemma · mradermacher
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
MO
MOSS Image Text Encoder Heretic
Moss · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningTool use
HA
Hathor_Unstable-L3-8B-v0.3
Hathor · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
BI
BioMistral 7B
Biomistral · BioMistral
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BO
Borealis 4b
Borealis · mradermacher
Params3.88B
Context128K
Q4_K_M2.5 GB
GGUF
QW
DS R1 Qwen3 8B ArliAI RpR v4 Small
Qwen · Lewdiculous
Params8.19B
Context128K
Q4_K_M5.0 GB
ReasoningRoleplay
OM
Omni R1
Omni · mradermacher
Params7B
Context5K
Q4_K_M4.3 GB
Reasoning
OR
Ornith 1.5 9B
Ornith · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
GE
Gemma 4 E4B IT ABLITERATED UNCENSORED PHILADELPHIA CLASS
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
Vision
MI
K2S3 Mistral 7b v1.46
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
SY
migtissera_synthia-7b-v3.0
Synthia · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
intel_neural-chat-7b-v3-1
Neural · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 4 E4B It The DECKARD Claude Opus Expresso Universe HERETIC UNCENSORED Thinking
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
CodingReasoningRoleplay
MI
Mistral Claude merged
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
LL
Dolphin Llama2 7B
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
YI
Yi-9B-Coder
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
Coding
QW
Qwen3 4B Instruct 2507 Abliterated
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
MI
mistral 7b orpo v5.0
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
Phi 3 song lyrics 1.0 i1
Phi · mradermacher
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
GE
gemma 4 E4B it qat q4 0 heretic
Gemma · FadedRedStar
Params7.46B
Context128K
Smallest0.1 GB
CodingVision
LL
Llama 3.1 8B Lexi Uncensored V2 Heretic
Llama · Spaceballs
Params8B
Context128K
Q4_K_M4.9 GB
GGUF

Other sizes

Keep browsing