Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
ME
Mephisto 1.0 4B
Mephisto · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
ME
Mephisto 1.5 4B
Mephisto · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
CodingReasoning
LF
LFM2.5 8B A1B APEX
Lfm · mudler
Params8.47B
Context125K
Smallest3.6 GB
GGUF
GE
GemmaTheWriter
Gemma · DavidAU
Params9.24B
Context8K
Q4_K_M5.8 GB
Roleplay
IN
Internlm2 Math 7b
InternLM · mradermacher
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
AT
N ATLaS
Atlas · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
llamas-community_llamaguard-7b
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
OpenChat 3.5 7B Qwen v2.0
Qwen · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
FO
Foundation Sec 8B Reasoning
Foundation · fdtn-ai
Params8B
Context128K
Smallest8.5 GB
Reasoning
MI
Ministral 3 8B Reasoning 2512 Esper3.1
Mistral · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
CodingReasoning
QW
Qwen Image 2.1 Turbo GGUFs
Qwen · realrebelai
Params7.12B
Context—
Smallest3.3 GB
GGUF
LL
Meta Llama 3 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 Mini 128k Instruct
Phi · second-state
Params3.82B
Context128K
Q4_K_M2.4 GB
Coding
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
CL
Claire 7B 0.1
Claire · TheBloke
Params7.22B
Context2K
Q4_K_M5.0 GB
GGUF
MI
Hermes-2-Pro-Mistral-7B-infinity
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
QW
QwenDGAF V3 4B
Qwen · hecan3d
Params4B
Context40K
Q4_K_M2.5 GB
Roleplay
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
IN
OpenChat 3.5 7B InternLM v2.0
InternLM · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MI
Mistral Syndicate 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
MiniCPM4.1 8B
MiniCPM · openbmb
Params8.2B
Context64K
Q4_K_M5.0 GB
GGUF
NE
Nehuronion Dvt 9b
Nehuronion · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
models--Groq--Llama-3-Groq-8B-Tool-Use
Llama · MaziyarPanahi
Params8B
Context8K
Q4_K_M4.9 GB
Tool use
FI
Fibo 2.0 edit turbo
Fibo · Abiray
Params8.23B
Context—
Q4_K_M5.3 GB
GGUF
LL
Llama 3.1 WhiteRabbitNeo 2 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
ZE
Zeta 2.1
Zeta · mradermacher
Params8.25B
Context32K
Q4_K_M5.1 GB
GGUF
QW
Huihui Qwen3.5 9B Claude 4.6 Opus Abliterated Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Reasoning
OP
openPangu Embedded 7B V1.1
Openpangu · mradermacher
Params8.03B
Context32K
Q4_K_M4.9 GB
Embedding
FA
Falcon H1 7B Instruct
Falcon · tiiuae
Params7.59B
Context256K
Q4_K_M4.6 GB
GGUF
EC
EchoVLM_v2
Echovlm · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision

Other sizes

Keep browsing