Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama 3.2 3B Instruct Abliterated
Llama · Hasaranga85
Params3.61B
Context128K
Q4_K_M2.2 GB
GGUF
GE
Locus Gemma 4 E2B
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
GGUF
LL
LightGPT 7B Llama2
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
BO
boto-9B
Boto · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
Llama 7b
Llama · QuantFactory
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Prettybird_Bce_Basic_Coder_8B
Qwen · mradermacher
Params7.6B
Context63K
Q4_K_M4.7 GB
Coding
GE
Gemma 2 9B ArliAI RPMax v1.1
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
KA
KAI 7B Instruct v0.1
Kai · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
IM
Z Anime
Image · SeeSee21
Params6.15B
Context—
Smallest4.5 GB
GGUF
LL
Tiamat 8b 1.2 Llama 3 DPO i1
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
KA
KangalKhan Alpha Emerald 7B
Kangalkhan · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
YI
Yi 1.5 9B Chat Abliterated
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
LL
CodeLlama 7B KStack clean
Llama · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
Mistral 7B Instruct v0.2 GGUF smashed
Mistral · PrunaAI
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
UM
Umievo Itr012 Gleipnir 7B
Umievo · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
KI
Kiel 2 VQA
Kiel · kiel2
Params7.6B
Context125K
Q4_K_M4.7 GB
Vision
MO
models--automerger--T3qm7xExperiment26-7B
MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
granite-8b-code-instruct
Granite · mradermacher
Params8.05B
Context4K
Q4_K_M4.9 GB
CodingTool use
LL
Llama-3-Luminurse-v0.1-OAS-8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
TR
Trion-M-7b
Trion · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
DE
OREAL DeepSeek R1 Distill Qwen 7B
DeepSeek · lmstudio-community
Params7.62B
Context128K
Q4_K_M4.7 GB
Reasoning
GP
GPT4chan 8B
Gpt · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
yeen214_llama2_7b_merge_orcafamily
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
SY
migtissera_synthia-7b-v1.5
Synthia · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Sakura Gemma4 E2B EmbeddingGemma2 DualMode
Gemma · webmp3
Params4.65B
Context128K
Smallest3.6 GB
Embedding
QW
Qwen2.5 7B Instruct 1M Thinking Claude Gemini GPT5.2 DISTILL
Qwen · mradermacher
Params7.62B
Context986K
Q4_K_M4.7 GB
ReasoningRoleplay
GL
GLM 4.6V Flash
GLM · mradermacher
Params9.4B
Context128K
Q4_K_M6.2 GB
ReasoningTool useVision
GR
Granite 8b Code Base 128k
Granite · mradermacher
Params8.05B
Context125K
Q4_K_M4.9 GB
Coding
MI
nousresearch_yarn-mistral-7b-64k
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other sizes

Keep browsing