Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
ZD
ZDTaichu5.0 9B
Zdtaichu · abenzerps
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
DI
DialFactSum ACU 8B
Dialfactsum · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
MI
MiniCPM3 4B
MiniCPM · openbmb
Params4.07B
Context32K
Q4_K_M2.5 GB
GGUF
QW
Qwen2 Audio 7B
Qwen · NexaAI
Params7.75B
Context8K
Q4_K_M4.8 GB
GGUF
KA
Kadoe 4B
Kadoe · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool useVision
MI
UCLA AGI Mistral7B PairRM SPPO Iter3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MP
Mpt 7b Chat
Mpt · mradermacher
Params6.65B
Context2K
Q4_K_M4.3 GB
GGUF
QW
Qwen3 4B
Qwen · NobodyWho
Params4B
Context32K
Q4_K_M2.5 GB
GGUF
WI
Winterreise-m7
Winterreise · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IN
models--automerger--Inex12Multiverseex26-7B
Inex · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GL
Poe 8B GLM5 Opus4.6 Sonnet4.5 Kimi Grok Gemini 3 Pro Preview HERETIC
GLM · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
SP
SpatialAxiom 9B
Spatialaxiom · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Vision
CO
beowolx_codeninja-1.0-openchat-7b
Codeninja · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
QW
MiRL Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
PH
Athene Phi 3.5 mini instruct orpo
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
LL
Aion RP Llama 3.1 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
ST
stabilityai japanese stablelm base gamma 7b
Stabilityai · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
EL
Elephant 1
Elephant · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
LL
Llama 2 7b Chat Hf Gpt 4 80k Base_Lora
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GU
Guanaco 7B Uncensored
Guanaco · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Fathom-Search-4B
Qwen · Mungert
Params4B
Context40K
Q4_K_M2.6 GB
GGUF
SK
SkyJM Edit 9B
Skyjm · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
HO
Sakura K2 Horizon 3.7B
Horizon · webmp3
Params5.06B
Context512K
Smallest2.2 GB
GGUF
MI
Nous Hermes 2 Mistral 7B DPO
Mistral · NousResearch
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
QW
Qwen3 8B Abliterated
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
LL
Llama3 8B Chinese Chat
Llama · Monor
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
SP
Spark X2.6 4B Uncensored
Spark · mradermacher
Params4.11B
Context1024K
Q4_K_M2.6 GB
ReasoningTool use
TR
bluenipples_trion-m-7b
Trion · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SW
Swallow-7b-NVE-instruct-hf
Swallow · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
NE
neo_7b_instruct_v0.1
Neo · legraphista
Params7.79B
Context8K
Smallest1.7 GB
GGUF

Other sizes

Keep browsing