Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
cognitivecomputations_dolphin-2.6-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
CodingTool use
GE
Huihui gemma 4 E2B it qat q4 0 unquantized abliterated
Gemma · huihui-ai
Params4.63B
Context128K
Smallest0.2 GB
Vision
ST
StartLux Decision v1 9B Q4 K M
Startlux · startlux-models
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
ID
ideogram 4
Ideogram · stduhpf
Params9.28B
Context—
Smallest3.6 GB
GGUF
ST
StartLux-Decision-v1.1-9B-Instant
Startlux · startlux-models
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
GE
Gemma 3 4b It Abliterated
Gemma · bartowski
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
GE
Gemma 3 4b It Qat Q4_0 Unquantized
Gemma · ggml-org
Params3.88B
Context128K
Smallest2.5 GB
Vision
QW
Qwen Image 2.1
Qwen · chfm
Params7.12B
Context—
Q4_K_M4.2 GB
GGUF
ON
OneJev 4B
Onejev · bartowski
Params4.84B
Context256K
Q4_K_M3.2 GB
Vision
LI
LightOnOCR 3 4B
Lightonocr · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
QW
Qwythos 9B Claude Mythos 5 1M Uncensored Heretic
Qwythos · llmfan46
Params8.95B
Context1024K
Q4_K_M6.2 GB
ReasoningTool useVision
QW
OmniCoder Qwen3.5 9B Claude 4.6 Opus Uncensored v2
Qwen · Ngixdev
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningVision
AP
Apertus-8B-Instruct-2509
Apertus · unsloth
Params8.05B
Context64K
Q4_K_M5.1 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct
Qwen · unsloth
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
QW
Qwen2.5-7B-Instruct-1M
Qwen · Mungert
Params7.6B
Context986K
Q4_K_M4.8 GB
GGUF
OR
orpheus 3b 0.1 ft Q4 K M
Orpheus · isaiahbjork
Params3.78B
Context128K
Q4_K_M2.4 GB
GGUF
NE
NeuralLemon 7B
Neurallemon · backyardai
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
FLUX 3 Image Klein TE Uncensored
Qwen · AtomicChat
Params4.02B
Context256K
Smallest2.5 GB
CodingTool useVision
QW
Qwen3 VL 4B Instruct C_Abliterated v2
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
CodingVision
QW
Satyr V0.1 4B
Qwen · PantheonUnbound
Params4B
Context256K
Q4_K_M2.5 GB
Reasoning
QU
Quyet 1.0 Medium
Quyet · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
Vision
LL
KISTI KONI KONI Llama3 8B Instruct 20240729
Llama · RichardErkhov
Params8B
Context—
Q4_K_M0.0 GB
GGUF
QW
Qwen3-Vl-4B-Thinking
Qwen · unsloth
Params4B
Context256K
Q4_K_M2.5 GB
ReasoningVision
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.8 4B Distilled
Qwen · Ma7ee7
Params4B
Context256K
Q4_K_M2.5 GB
Reasoning
OL
olmOCR 2 7B 1025
OLMo · bartowski
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
QW
Qwen3.5 9B Uncensored HauhauCS Aggressive Q4 K M
Qwen · Manojb
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
QW
RuadaptQwen3 4B Instruct
Qwen · RefalMachine
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
LE
lev
Lev · ggml-org
Params4.2B
Context256K
Q4_K_M3.0 GB
GGUF
QW
Qwen3.5 9B Uncensored
Qwen · bowmanslayer
Params9.2B
Context256K
Q4_K_M5.8 GB
GGUF

Other sizes

Keep browsing