Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
ZD
ZDTaichu5.0 9B
Zdtaichu · abenzerps
Params8.95B
Context256K
Q4_K_M5.6 GB
DI
DialFactSum ACU 8B
Dialfactsum · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
MI
MiniCPM3 4B
MiniCPM · openbmb
Params4.07B
Context32K
Q4_K_M2.5 GB
QW
Qwen2 Audio 7B
Qwen · NexaAI
Params7.75B
Context8K
Q4_K_M4.8 GB
KA
Kadoe 4B
Kadoe · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
MI
UCLA AGI Mistral7B PairRM SPPO Iter3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MP
Mpt 7b Chat
Mpt · mradermacher
Params6.65B
Context2K
Q4_K_M4.3 GB
QW
Qwen3 4B
Qwen · NobodyWho
Params4B
Context32K
Q4_K_M2.5 GB
WI
Winterreise-m7
Winterreise · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
IN
models--automerger--Inex12Multiverseex26-7B
Inex · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GL
Poe 8B GLM5 Opus4.6 Sonnet4.5 Kimi Grok Gemini 3 Pro Preview HERETIC
GLM · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
SP
SpatialAxiom 9B
Spatialaxiom · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
CO
beowolx_codeninja-1.0-openchat-7b
Codeninja · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
QW
MiRL Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
PH
Athene Phi 3.5 mini instruct orpo
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.3 GB
LL
Aion RP Llama 3.1 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
ST
stabilityai japanese stablelm base gamma 7b
Stabilityai · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
EL
Elephant 1
Elephant · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Llama 2 7b Chat Hf Gpt 4 80k Base_Lora
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GU
Guanaco 7B Uncensored
Guanaco · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Fathom-Search-4B
Qwen · Mungert
Params4B
Context40K
Q4_K_M2.6 GB
SK
SkyJM Edit 9B
Skyjm · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
HO
Sakura K2 Horizon 3.7B
Horizon · webmp3
Params5.06B
Context512K
Smallest2.2 GB
MI
Nous Hermes 2 Mistral 7B DPO
Mistral · NousResearch
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 8B Abliterated
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
LL
Llama3 8B Chinese Chat
Llama · Monor
Params8B
Context8K
Q4_K_M4.9 GB
SP
Spark X2.6 4B Uncensored
Spark · mradermacher
Params4.11B
Context1024K
Q4_K_M2.6 GB
TR
bluenipples_trion-m-7b
Trion · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
SW
Swallow-7b-NVE-instruct-hf
Swallow · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
NE
neo_7b_instruct_v0.1
Neo · legraphista
Params7.79B
Context8K
Smallest1.7 GB