Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Qwable 9B Claude Fable 5
Qwable · empero-ai
Params8.95B
Context256K
Q4_K_M5.6 GB
NE
NVIDIA Nemotron 3.5 Bolt E4B Pollard
Nemotron · PollardWeights
Params7.52B
Context128K
Q4_K_M0.2 GB
QW
Huihui Qwen3 4B Abliterated v2
Qwen · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
AI
Airoboros L2 7B 2.2
Airoboros · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen3 4B Thinking 2507
Qwen · bartowski
Params4B
Context256K
Q4_K_M2.5 GB
HU
Hunyuan MT 7B
Hunyuan · imikeliu
Params7.5B
Context256K
Q4_K_M4.6 GB
LL
4 alokk llama 3 8b English to Hinglish
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen2.5 VL 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
DE
DeepSeek R1 Distill Qwen 7B Uncensored
DeepSeek · mradermacher
Params7.61B
Context128K
Q4_K_M4.7 GB
CH
Chroma1 Flash
Chroma · silveroxides
Params8.9B
Context—
Q4_K_M5.6 GB
MA
MangaLMM
Mangalmm · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
LL
dolphin 2.9 llama3 8b
Llama · dphn
Params8B
Context8K
Q4_K_M4.9 GB
CL
Clef Flash Heretic
Clef · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
GELab-Zero-4B-preview
Qwen · Mungert
Params4B
Context256K
Q4_K_M2.6 GB
QW
Qwen3 Vl 8b Heretic 1.3.0
Qwen · DreamFast
Params8.19B
Context256K
Q4_K_M5.0 GB
IM
z image turbo adreno
Image · zhiyuanasad
Params6.15B
Context—
Smallest2.3 GB
DE
Deepseek Coder 6.7b Base
DeepSeek · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
TI
Tini Cybersec 8B A1B
Tini · bartowski
Params8.47B
Context125K
Q4_K_M5.2 GB
HO
K2-Horizon-7B
Horizon · bartowski
Params9B
Context512K
Q4_K_M5.8 GB
SP
Spark X2.5 4B
Spark · ngquocvinh
Params4.1B
Context1024K
Q4_K_M2.6 GB
MI
Mistral 7B v0.1
Mistral · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
dolphin-llama3-zh-cn-uncensored
Llama · stephenlzc
Params8B
Context8K
Q4_K_M4.9 GB
PY
Pyrex 8B Instruct Uncensored i1
Pyrex · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GU
GUI Owl 1.5 4B Instruct
Gui · mradermacher
Params4B
Context32K
Q4_K_M2.5 GB
GD
GDP 4B
Gdp · mradermacher
Params4.02B
Context256K
Q4_K_M2.5 GB
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
GE
gemma-1.1-7b-it
Gemma · bartowski
Params8.54B
Context8K
Q4_K_M5.3 GB
FL
Sakura FLUX.3 Image Klein
Flux · webmp3
Params5.55B
Context—
Q4_K_M3.3 GB
JA
stabilityai_japanese-stablelm-instruct-gamma-7b
Japanese · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
silicon based girlfriend v2
Qwen · RX5950XT
Params9.2B
Context256K
Q4_K_M5.8 GB