Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
AM
Amaretto 8B
Amaretto · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
Tool use
TE
tenyx_tenyxchat-7b-v1
Tenyxchat · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
PY
Pythia 6.9b Deduped
Pythia · mradermacher
Params6.86B
Context2K
Q4_K_M4.4 GB
GGUF
HA
Haru Kanana 8b
Haru · mradermacher
Params8B
Context32K
Q4_K_M4.9 GB
Roleplay
YI
Yi-1.5-9B-Chat-16K
Yi · mradermacher
Params8.83B
Context16K
Q4_K_M5.3 GB
GGUF
ST
L3-8B-Stheno-v3.2
Stheno · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
meta-math_metamath-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ZA
Zarafusionex 1.1 L2 7B
Zarafusionex · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Jan-v2-VL-high
Qwen · Mungert
Params8.2B
Context256K
Q4_K_M5.4 GB
ReasoningVision
LO
LongCat Image
Longcat · vantagewithai
Params6.27B
Context—
Q4_K_M3.7 GB
GGUF
VI
vibevoice asr
Vibevoice · cstr
Params8.33B
Context—
Smallest0.0 GB
GGUF
TO
TowerInstruct 7B v0.1
Towerinstruct · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen3 8B Heretic
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
GE
Gemma 3 it 4B Uncensored DBL X
Gemma · DavidAU
Params4.26B
Context128K
Q4_K_M2.7 GB
Roleplay
YI
Hermes-2.5-Yi-1.5-9B-Chat
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
Tool use
QW
dolphin-2.9.2-qwen2-7b
Qwen · dphn
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
SP
SpydazWebAI_Human_AGI
Spydazwebai · mradermacher
Params7.24B
Context512K
Q4_K_M4.4 GB
Coding
QW
Qwythos 9B v2
Qwythos · mradermacher
Params9.2B
Context1024K
Q4_K_M5.8 GB
ReasoningVision
QW
Qwen3 VL 8B Abliterated Caption It
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
GE
gemma 4 e2b yemeni arabic assistant
Gemma · yemeni-ai-lab
Params4.63B
Context128K
Q4_K_M3.4 GB
Vision
AH
Ahma 7B
Ahma · mradermacher
Params7B
Context2K
Q4_K_M4.3 GB
GGUF
MI
Infinity Instruct 7M 0729 Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Google_Gemma 4 E4B It Qat Q4_0 Unquantized
Gemma · lmstudio-community
Params7.46B
Context128K
Smallest5.2 GB
Vision
CA
CantoneseLLM v2.0 8B Thinking Chat Vector Merged
Cantonesellm · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
Reasoning
IN
Index Nailong 9B
Index · mradermacher
Params8.95B
Context224K
Q4_K_M5.6 GB
GGUF
VI
T VisStar 7B v0.1 i1
Visstar · mradermacher
Params7.29B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.8 27B Escha W2
Qwen · aj9o9
Params6.33B
Context256K
Smallest2.9 GB
GGUF
VI
Vietnamese 4B Chat abliterated
Vietnamese · N14QM
Params4.2B
Context256K
Q4_K_M2.7 GB
Vision
QW
Huihui Qwen3.5 9B Claude 4.6 Opus Abliterated Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Reasoning
LL
Hermes 2 Theta Llama 3 8B
Llama · NousResearch
Params8B
Context8K
Q4_K_M4.9 GB
Tool use

Other sizes

Keep browsing