Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
SN
SniffyOtter 7B
Sniffyotter · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
SW
Swallow-7b-NVE-hf
Swallow · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.5 9B EmperoAI Qwen3.8 Distill Heretic CyberStrike
Qwen · Aapolon
Params9.2B
Context256K
Q4_K_M5.8 GB
Vision
QW
Nyx RP 9B Instruct 2608 v1
Qwen · Indexnusrefather
Params9.2B
Context256K
Q4_K_M5.8 GB
Roleplay
LL
saraprice llama2 7B backdoor DEPLOYMENT
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Hermes-2-Pro-Mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
TR
TranslatePsy AfriSLM 4B
Translatepsy · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
ReasoningTool use
MI
lemonilia_ashhlimarp-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama Leonida
Llama · mradermacher
Params8.2B
Context128K
Q4_K_M5.0 GB
Tool use
ME
models--automerger--T3qm7Mergerix-7B
Mergerix · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LF
LFM2.5 8B A1B heretic
Lfm · saidutta69
Params8.47B
Context125K
Q4_K_M5.2 GB
ReasoningRoleplayTool use
AN
Andromeda 8B Coder i1
Andromeda · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
CodingTool use
YA
models--automerger--Experiment28Yam-7B
Yam · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
EV
EAN 7B i1
Evan · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Reasoning
SU
SummLlama3 8B i1
Summllama · mradermacher
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3-4B-128K
Qwen · unsloth
Params4B
Context128K
Q4_K_M2.5 GB
GGUF
TI
Tiamat-7b-1.1-DPO
Tiamat · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama-3-Patronus-Lynx-8B-Instruct
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
RO
ROMA 7B
Roma · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
meta-llama-7b
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
KA
KAI 7B v0.1
Kai · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
OX
OxCoder 9B
Oxcoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingVision
QW
Qwen3.8 9B Heretic Uncensored
Qwen · optionalAI
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
HO
Holo 3.1 9B
Holo · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
QW
Remnant Qwen3 8b
Qwen · bartowski
Params8.19B
Context32K
Q4_K_M5.0 GB
Roleplay
HE
L3.1 RP Hero Dirty Harry 8B
Hero · DavidAU
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
VI
Vietnamese 4B Chat Abliterated
Vietnamese · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
CA
calme-2.3-legalkit-8b
Calme · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Liberation Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
QW
CaptchaKraken v1.2
Qwen · CaptchaKraken
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision

Other sizes

Keep browsing