Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
SI
SILMA 9B Instruct v1.0
Silma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
AV
Average_Normie_v3.69_8B
Average · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GE
Gemma 4 E4B Abliterated Uncensored
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
Roleplay
BA
internistai base 7b v0.2
Base · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NU
NuMarkdown-8B-Thinking
Numarkdown · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
ReasoningVision
GE
gemma 4 E2B it QAT
Gemma · lmstudio-community
Params4.63B
Context128K
Smallest3.3 GB
Vision
QW
Qwen2.5 7B Instruct
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
NE
intel_neural-chat-7b-v3-2
Neural · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FA
Falcon3 7B Instruct Abliterated
Falcon · bartowski
Params7.46B
Context32K
Q4_K_M4.6 GB
GGUF
WR
Wraith 8b
Wraith · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
ReasoningRoleplay
LL
cyber risk llama 3 8b
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
Vision
MU
Muslim 6B v3
Muslim · mradermacher
Params5.94B
Context256K
Q4_K_M3.7 GB
Tool use
GE
Gemma 4 E4B It GMCE
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
GGUF
NE
Next2.5
Next · thelamapi
Params4.2B
Context256K
Q4_K_M2.7 GB
ReasoningVision
DQ
dqnMath v1
Dqnmath · mradermacher
Params4.02B
Context256K
Q4_K_M2.5 GB
ReasoningTool use
PY
Pyrz 8B Instruct Uncensored i1
Pyrz · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
godot qwen2.5 coder 7b instruct bnb 4bit
Qwen · bekoozkan
Params7.6B
Context32K
Q4_K_M4.7 GB
Coding
OR
Deepreinforce ai_Ornith 1.0 9B
Ornith · lmstudio-community
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
OZ
Ozyvler Ozygram Neural 7B Coder MCTS
Ozyvler · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
GE
Gemma2 9b Neuro sama Q4 K M
Gemma · neifuisan
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LO
Loopl Vl 4b
Loopl · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
Tool useVision
EU
EuroLLM 9B MeditronFO
Eurollm · mradermacher
Params9.15B
Context4K
Q4_K_M5.6 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
Starling LM 7B beta
Starling · QuantFactory
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MI
Mahou-1.3a-mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DA
abideen_darevox-7b
Darevox · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
EL
electric mist 7b
Electric · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
YI
Yi-6B-200K
Yi · duyntnet
Params6.06B
Context195K
Q4_K_M3.7 GB
GGUF
QW
Qwen3.5 4B Instruct
Qwen · Hob-forge
Params4.33B
Context256K
Q4_K_M2.8 GB
GGUF
AG
L3.1 Aglow Vulca v0.1 8B i1
Aglow · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay

Other sizes

Keep browsing