Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
ID
ideogram 4
Ideogram · leejet
Params9.28B
Context—
Smallest5.6 GB
GGUF
BE
BetterWright K2 Horizon 7B Uno
Betterwright · mradermacher
Params9B
Context512K
Q4_K_M5.6 GB
Tool use
LL
Llama3.1 8B ShiningValiant2
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
AV
Avrik Lite 4B
Avrik · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
GGUF
GE
GELab Zero 4B Preview
Gelab · bartowski
Params4B
Context256K
Q4_K_M2.5 GB
Vision
MA
Magicoder S DS 6.7B
Magicoder · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
QW
Qwen3.5 9b Claude 4.8 Opus reasoning
Qwen · TheCidSama
Params9.2B
Context256K
Smallest5.8 GB
CodingReasoning
DO
weyaxi_dolphin2.1-openorca-7b
Dolphin · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama-2-7b-WikiChat-fused
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
CodeLlama 7b Instruct Hf
Llama · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
QW
Video Companions
Qwen · off-grid-ai
Params4.02B
Context256K
Q4_K_M2.5 GB
CodingTool useVision
LL
rombos Llama3 8B Instruct Replete Adapted i1
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
CodingTool use
QW
Qwen2.5 Coder 7B
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
Coding
MI
nondzu_mistral-7b-code-16k-qlora
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
OR
Ornith 1.5 9B Heretic
Ornith · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
KE
Kev-4B Artemis 2.5GB
Kev · turintech
Params4.21B
Context256K
Q4_K_M2.5 GB
ReasoningTool use
QW
Qwen3.5 4B Heretic 2xMerge DavidAU
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
Reasoning
LL
LosslessMegaCoder Llama2 7B Mini
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
Coding
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · legraphista
Params8B
Context128K
Smallest2.0 GB
RoleplayTool use
SW
Swallow-7b-instruct-v0.1
Swallow · mradermacher
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct Heretic
Qwen · saidutta69
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
LL
Hermes 3 Llama 3.1 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
RoleplayTool use
GE
Gemma 3 4b It
Gemma · DavidAU
Params3.88B
Context128K
Q4_K_M3.3 GB
GGUF
HO
K2 Horizon 7B
Horizon · NANI-Nithin
Params9B
Context512K
Q4_K_M5.6 GB
GGUF
OX
OxCoder 9B GMCE
Oxcoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningTool use
UM
umiyuki Japanese Chat Umievo itr001 7b
Umiyuki · mmnga
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
GR
Granite 7b Base
Granite · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
EX
models--automerger--Experiment28Experiment29-7B
Experiment · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LI
T Lite It 1.0
Lite · aovchinnikov
Params7.6B
Context32K
Q4_K_M4.7 GB
GGUF

Other sizes

Keep browsing