Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · mlabonne
Params8B
Context128K
Q4_K_M4.9 GB
QW
MiMo V2.6 Distill Qwen 9B Ablitrated
Qwen · Hikari07jp
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen_Qwen3 VL 4B Instruct
Qwen · lmstudio-community
Params4B
Context256K
Q4_K_M2.5 GB
QW
Sakura Qwen Image 2.1 Turbo Uncensored
Qwen · webmp3
Params7.12B
Context—
Q4_K_M4.1 GB
AC
xl_regrind_v1
Acestep · mdmachine
Params4.99B
Context32K
Q4_K_M3.0 GB
FL
FLUX 3 Image Klein GGUFs
Flux · realrebelai
Params5.54B
Context—
Q4_K_M3.4 GB
FB
fblgit juanako 7b UNA
Fblgit · RichardErkhov
Params7.24B
Context—
Q4_K_M0.0 GB
QW
Qwen3 4b Z Image Turbo AbliteratedV1
Qwen · BennyDaBall
Params4B
Context40K
Q4_K_M2.5 GB
QW
Qwen3.8 9B heretic uncensored NVFP4
Qwen · Noobito45
Params8.95B
Context256K
Q4_K_M5.2 GB
MI
mistralai_mistral-7b-instruct-v0.1
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 8B
Qwen · bartowski
Params8.19B
Context32K
Q4_K_M5.0 GB
QW
Qwen3.5 9B Abliterated
Qwen · Abiray
Params8.95B
Context256K
Q4_K_M5.6 GB
OR
Ornith 1.5 9B UNCENSORED
Ornith · dealignai
Params8.95B
Context256K
Q4_K_M5.6 GB
PH
Phi 3.5 Mini Instruct
Phi · bartowski
Params3.82B
Context128K
Q4_K_M2.4 GB
LL
Dolphin 3.0 Llama 3.1 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
ME
Medgemma-4B-It
Medgemma · unsloth
Params3.88B
Context128K
Q4_K_M2.5 GB
CY
z image cyberrealistic v9
Cyberrealistic · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
FL
FLUX.2 klein 9B
Flux · leejet
Params9.08B
Context—
Smallest5.6 GB
QW
Qwen3VL 8B Uncensored HauhauCS Aggressive
Qwen · HauhauCS
Params8.2B
Context256K
Q4_K_M5.0 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · lmstudio-community
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Qwen2.5 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
PR
Fish Speech S2 Pro Dual AR
Pro · rodrigomt
Params4.95B
Context32K
Q4_K_M3.6 GB
HO
K2 Horizon 7B
Horizon · IFM
Params9B
Context512K
Q4_K_M5.6 GB
QW
Qwen2.5 7B Instruct abliterated v2
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
FL
Flux2 Klein 9B True V3
Flux · wikeeyang
Params9.08B
Context—
Smallest5.7 GB
QW
Qwen3.5 Text 9B NSFW RP RolePlay
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Meta-Llama-3-8B-Instruct
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen3 8B Abliterated
Qwen · bartowski
Params8.19B
Context40K
Q4_K_M5.0 GB
QW
LFM2.5 8B A1B
Qwen · DavidAU
Params8.47B
Context125K
Smallest4.6 GB
QW
Qwen3.5 9B Ultra Uncensored Heretic
Qwen · llmfan46
Params8.95B
Context256K
Q4_K_M5.5 GB