Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
ME
Mephisto 1.0 4B
Mephisto · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ME
Mephisto 1.5 4B
Mephisto · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
LF
LFM2.5 8B A1B APEX
Lfm · mudler
Params8.47B
Context125K
Smallest3.6 GB
GE
GemmaTheWriter
Gemma · DavidAU
Params9.24B
Context8K
Q4_K_M5.8 GB
IN
Internlm2 Math 7b
InternLM · mradermacher
Params7.74B
Context8K
Q4_K_M4.7 GB
AT
N ATLaS
Atlas · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
LL
llamas-community_llamaguard-7b
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
QW
OpenChat 3.5 7B Qwen v2.0
Qwen · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
FO
Foundation Sec 8B Reasoning
Foundation · fdtn-ai
Params8B
Context128K
Smallest8.5 GB
MI
Ministral 3 8B Reasoning 2512 Esper3.1
Mistral · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
QW
Qwen Image 2.1 Turbo GGUFs
Qwen · realrebelai
Params7.12B
Context—
Smallest3.3 GB
LL
Meta Llama 3 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
PH
Phi 3 Mini 128k Instruct
Phi · second-state
Params3.82B
Context128K
Q4_K_M2.4 GB
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
CL
Claire 7B 0.1
Claire · TheBloke
Params7.22B
Context2K
Q4_K_M5.0 GB
MI
Hermes-2-Pro-Mistral-7B-infinity
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
QwenDGAF V3 4B
Qwen · hecan3d
Params4B
Context40K
Q4_K_M2.5 GB
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
IN
OpenChat 3.5 7B InternLM v2.0
InternLM · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
MI
Mistral Syndicate 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
MiniCPM4.1 8B
MiniCPM · openbmb
Params8.2B
Context64K
Q4_K_M5.0 GB
NE
Nehuronion Dvt 9b
Nehuronion · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
LL
models--Groq--Llama-3-Groq-8B-Tool-Use
Llama · MaziyarPanahi
Params8B
Context8K
Q4_K_M4.9 GB
FI
Fibo 2.0 edit turbo
Fibo · Abiray
Params8.23B
Context—
Q4_K_M5.3 GB
LL
Llama 3.1 WhiteRabbitNeo 2 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
ZE
Zeta 2.1
Zeta · mradermacher
Params8.25B
Context32K
Q4_K_M5.1 GB
QW
Huihui Qwen3.5 9B Claude 4.6 Opus Abliterated Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
OP
openPangu Embedded 7B V1.1
Openpangu · mradermacher
Params8.03B
Context32K
Q4_K_M4.9 GB
FA
Falcon H1 7B Instruct
Falcon · tiiuae
Params7.59B
Context256K
Q4_K_M4.6 GB
EC
EchoVLM_v2
Echovlm · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB