Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
GE
Huihui gemma 4 E4B it qat q4 0 unquantized abliterated
Gemma · huihui-ai
Params7.46B
Context128K
Smallest0.2 GB
Vision
QW
Qwen Image 2.1 GGUFs
Qwen · realrebelai
Params7.12B
Context—
Smallest4.1 GB
GGUF
FL
flux2 klein 9b uncensored text encoder
Flux · ponpoke
Params8.19B
Context—
Q4_K_M5.0 GB
Coding
QW
Qwen3.5 4B Uncensored LocalAI
Qwen · weareapexcreators
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
HU
Hunyuan MT 7B
Hunyuan · mradermacher
Params7.5B
Context256K
Q4_K_M4.6 GB
GGUF
NE
NVIDIA Nemotron Labs Dory1.0 4B
Nemotron · nvidia
Params3.98B
Context256K
Smallest3.7 GB
ReasoningTool use
BA
BarryFutureman NeuralTurdusVariant1 7B
Barryfutureman · RichardErkhov
Params7B
Context—
Q4_K_M0.0 GB
GGUF
GE
Gemma-3-4B-It-Qat
Gemma · unsloth
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
RO
L3.2 Rogue Creative Instruct Uncensored Abliterated 7B
Rogue · DavidAU
Params7.53B
Context128K
Q4_K_M4.6 GB
Roleplay
HO
K2 Horizon 3.7B
Horizon · IFM
Params5.06B
Context512K
Q4_K_M3.2 GB
GGUF
QW
Qwen3 VL 8B Instruct Abliterated v2
Qwen · prithivMLmods
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
ST
StartLux-Decision-v1.1-4B-Instant
Startlux · startlux-models
Params4.2B
Context256K
Q4_K_M2.7 GB
Vision
QW
OrcaRouter Qwen3.8 Flash Next Uncensored NVFP4 GPTQ Strata
Qwen · Maximilian228
Params4.95B
Context—
Smallest1.3 GB
GGUF
MO
MOSS Image SFT
Moss · leejet
Params5.08B
Context—
Smallest2.2 GB
GGUF
FL
vllm-d3-flash
Flash · ggml-org
Params8.95B
Context256K
Q4_K_M6.6 GB
Vision
QW
Qwen3.8 9B Distill uncensored heretic Q4 K M
Qwen · ge525
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
MO
moshiko candle q8
Moshiko · kyutai
Params7.69B
Context—
Smallest8.2 GB
GGUF
ST
L3-8B-Stheno-v3.2
Stheno · Lewdiculous
Params8B
Context8K
Q4_K_M4.9 GB
Roleplay
LL
VAGOsolutions Llama 3 SauerkrautLM 70b Instruct
Llama · RichardErkhov
Params6B
Context—
Q4_K_M0.0 GB
GGUF
QW
Qwen3.5 9B Claude 4.6 Opus Reasoning Distilled
Qwen · Jackrong
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
QW
Atomic Chat / Qwen3.5 9B
Qwen · AtomicChat
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
SP
Spark X2.6 4B
Spark · bartowski
Params4.11B
Context1024K
Q4_K_M2.6 GB
GGUF
ST
StartLux-Decision-v1.1-9B-AutoThink
Startlux · startlux-models
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
GE
gemma 4 E4B it uncensored
Gemma · TrevorJS
Params7.5B
Context128K
Q4_K_M5.3 GB
GGUF
NA
Nanbeige4.2 3B
Nanbeige · bartowski
Params4.17B
Context256K
Q4_K_M2.7 GB
GGUF
ST
StartLux Decision v1 9B Q8 0
Startlux · startlux-models
Params8.95B
Context256K
Smallest9.5 GB
GGUF
WI
ehartford_wizardlm-7b-uncensored
Wizardlm · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
LL
DarkIdol Llama 3.1 8B Instruct 1.2 Uncensored
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
GL
GLM 4.6V Flash
GLM · ggml-org
Params9.4B
Context128K
Q4_K_M6.2 GB
Vision
QW
Atomic Chat / Qwen3.5 4B
Qwen · AtomicChat
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use

Other sizes

Keep browsing