Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
CO
Yi Coder 9B Chat
Coder · mradermacher
Params8.83B
Context128K
Q4_K_M5.3 GB
GL
Glm 4 9b Chat
GLM · bartowski
Params9.4B
Context128K
Q4_K_M6.3 GB
QW
Qwen2.5 7B Instruct 1M
Qwen · bartowski
Params7.62B
Context986K
Q4_K_M4.7 GB
PO
pony v7 base
Pony · purplesmartai
Params6.86B
Context—
Smallest3.9 GB
QW
Darrellbest Qwen Image 2.1 PE I2I Heretic
Qwen · darrellbest
Params8.95B
Context256K
Q4_K_M5.6 GB
MI
llava 1.6 mistral 7b
Mistral · cjpais
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Huihui Qwen3.5 4B Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
YA
YandexGPT 5 Lite 8B Instruct
Yandexgpt · yandex
Params8B
Context32K
Q4_K_M4.9 GB
CO
comfyui full pack
Comfyui · LuckyOda
Params4B
Context40K
Smallest4.3 GB
AY
Aya Expanse 8b
Aya · bartowski
Params8B
Context8K
Q4_K_M5.1 GB
MI
Mistral 7B Instruct v0.3
Mistral · QuantFactory
Params7.25B
Context32K
Q4_K_M4.4 GB
DE
Deepseek-Prover-V2-7B
DeepSeek · unsloth
Params6.91B
Context32K
Q4_K_M4.2 GB
SP
Spark X2.6 4B LocalAI
Spark · weareapexcreators
Params4.45B
Context1024K
Q4_K_M3.5 GB
GE
Gemma 4 E4B
Gemma · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
QW
Qwen 3.5 4B RP Finetune
Qwen · minas2025
Params4.2B
Context256K
Smallest4.5 GB
PR
PreciseCoder 9B
Precisecoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GE
gemma-2-9b-it
Gemma · lmstudio-community
Params9.24B
Context8K
Q4_K_M5.8 GB
HU
HunyuanVideo 1.5 T2V 480p
Hunyuanvideo · jayn7
Params8.33B
Context—
Q4_K_M5.1 GB
GR
grok oss Apollyon 8B
Grok · c4tdr0ut
Params8B
Context128K
Smallest5.7 GB
LY
Lythri-7B-A4B
Lythri · Lythri
Params7.46B
Context128K
Smallest5.7 GB
LL
dolphin 2.9.4 llama3.1 8b
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
RW
Rwkv 6 World 7b
Rwkv · bartowski
Params7.64B
Context1024K
Q4_K_M4.8 GB
LF
LFM2.5 8B A1B Hermes Agentic Coder Abliterated v3
Lfm · DuoNeural
Params8.47B
Context125K
Q4_K_M5.2 GB
FI
FinanceGemma E4B
Financegemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
VE
Veloxy Flash 15B
Veloxy · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
FA
Falcon H1R 7B
Falcon · tiiuae
Params7.59B
Context256K
Q4_K_M4.6 GB
YI
Yi-1.5-6B-Chat
Yi · bartowski
Params6.06B
Context4K
Q4_K_M3.7 GB
LL
llm4decompile 9b v2
Llm · mradermacher
Params8.83B
Context128K
Q4_K_M5.3 GB
MI
Ministral 3 8B Reasoning 2512
Mistral · mistralai
Params8.49B
Context256K
Q4_K_M5.2 GB
GE
Huihui Gemma 4 E4B It Abliterated
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB