Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
DR
Drex v1.5
Drex · nace-ai
Params9B
Context256K
Smallest9.5 GB
GGUF
LL
tap-m_luna-ai-llama2-uncensored
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
GR
Granite-4.0-H-Tiny
Granite · unsloth
Params6.94B
Context1024K
Q4_K_M4.3 GB
GGUF
QW
Qwen3 VL 8B Instruct Uncensored V2
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
MI
dolphin 2.8 mistral 7b v02
Mistral · lmstudio-community
Params7.24B
Context32K
Q4_K_M4.4 GB
CodingTool use
PY
Pyrut 8B Instruct Uncensored i1
Pyrex · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
RO
RoLlama2-7b-Instruct
Rollama · legraphista
Params6.74B
Context4K
Smallest1.5 GB
GGUF
GE
gemma 3 4b it qat q4 0
Gemma · google
Params3.88B
Context—
Smallest3.2 GB
Vision
GE
Gemma 4 E4B Abliterated Uncensored
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
Roleplay
FL
FLUX.2 Klein 9B KV
Flux · QuantStack
Params9.08B
Context—
Q4_K_M5.7 GB
GGUF
HY
Hy MT2 7B
mradermacher
Params7.5B
Context256K
Q4_K_M4.6 GB
GGUF
QW
Qwen3VL 8B Uncensored HauhauCS Balanced
Qwen · HauhauCS
Params8.2B
Context256K
Q4_K_M5.0 GB
Vision
UM
BojroPromptMaster uncensored v2 8B
Umbral · bojrodev
Params8B
Context8K
Q4_K_M4.9 GB
Roleplay
QW
Qwen3.5 9B BOS Q8 NonThinking
Qwen · shaban2024
Params8.95B
Context256K
Smallest9.5 GB
ReasoningTool use
PH
Phi 4 Mini Instruct
Phi · lmstudio-community
Params3.84B
Context128K
Q4_K_M2.5 GB
GGUF
TE
Ternary Bonsai 4B
Ternary · prism-ml
Params4B
Context32K
Smallest1.1 GB
GGUF
QW
MiMo V2.6 Distill Qwen 9B heretic
Qwen · saidutta69
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningRoleplay
HE
0 hero Matter 0.2 7B
Hero · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
llama3.1 airoboros3.2 QDT 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
AR
AREX Turbo
Arex · bartowski
Params4.2B
Context256K
Q4_K_M2.9 GB
ReasoningTool useVision
LF
LFM2.5-8B-A1B-Uncensored-Gaston
Lfm · gaston-parravicini
Params8.47B
Context125K
Q4_K_M5.2 GB
Reasoning
MI
MegaBeam Mistral 7B 512k
Mistral · bartowski
Params7.24B
Context512K
Q4_K_M4.4 GB
GGUF
JE
Jebadiah 9b v2
Jebadiah · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
Vision
MI
ehartford_dolphin-2.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
DeepSeek R1 Distill Llama 8B
DeepSeek · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Reasoning
GE
gemma4 coding agent
Gemma · S4MPL3BI4S
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
LL
Llm Jp 4 8b Instruct
Llm · mmnga-o
Params8.59B
Context64K
Q4_K_M5.3 GB
GGUF
NE
Nemotron 3.5 Bolt E4B GSQ RCO
Nemotron · abenzerps
Params7.52B
Context128K
Smallest0.4 GB
ReasoningTool useVision
CA
CAT Thinking 8B
Cat · mmnga-o
Params8.19B
Context40K
Q4_K_M5.0 GB
Reasoning
IN
InternLM2
InternLM · legraphista
Params7.74B
Context8K
Smallest1.9 GB
GGUF

Other sizes

Keep browsing