Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
DR
Drex v1.5
Drex · nace-ai
Params9B
Context256K
Smallest9.5 GB
LL
tap-m_luna-ai-llama2-uncensored
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GR
Granite-4.0-H-Tiny
Granite · unsloth
Params6.94B
Context1024K
Q4_K_M4.3 GB
QW
Qwen3 VL 8B Instruct Uncensored V2
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
MI
dolphin 2.8 mistral 7b v02
Mistral · lmstudio-community
Params7.24B
Context32K
Q4_K_M4.4 GB
PY
Pyrut 8B Instruct Uncensored i1
Pyrex · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
RO
RoLlama2-7b-Instruct
Rollama · legraphista
Params6.74B
Context4K
Smallest1.5 GB
GE
gemma 3 4b it qat q4 0
Gemma · google
Params3.88B
Context—
Smallest3.2 GB
GE
Gemma 4 E4B Abliterated Uncensored
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
FL
FLUX.2 Klein 9B KV
Flux · QuantStack
Params9.08B
Context—
Q4_K_M5.7 GB
HY
Hy MT2 7B
mradermacher
Params7.5B
Context256K
Q4_K_M4.6 GB
QW
Qwen3VL 8B Uncensored HauhauCS Balanced
Qwen · HauhauCS
Params8.2B
Context256K
Q4_K_M5.0 GB
UM
BojroPromptMaster uncensored v2 8B
Umbral · bojrodev
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen3.5 9B BOS Q8 NonThinking
Qwen · shaban2024
Params8.95B
Context256K
Smallest9.5 GB
PH
Phi 4 Mini Instruct
Phi · lmstudio-community
Params3.84B
Context128K
Q4_K_M2.5 GB
TE
Ternary Bonsai 4B
Ternary · prism-ml
Params4B
Context32K
Smallest1.1 GB
QW
MiMo V2.6 Distill Qwen 9B heretic
Qwen · saidutta69
Params8.95B
Context256K
Q4_K_M5.6 GB
HE
0 hero Matter 0.2 7B
Hero · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
llama3.1 airoboros3.2 QDT 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
AR
AREX Turbo
Arex · bartowski
Params4.2B
Context256K
Q4_K_M2.9 GB
LF
LFM2.5-8B-A1B-Uncensored-Gaston
Lfm · gaston-parravicini
Params8.47B
Context125K
Q4_K_M5.2 GB
MI
MegaBeam Mistral 7B 512k
Mistral · bartowski
Params7.24B
Context512K
Q4_K_M4.4 GB
JE
Jebadiah 9b v2
Jebadiah · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
MI
ehartford_dolphin-2.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
DeepSeek R1 Distill Llama 8B
DeepSeek · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GE
gemma4 coding agent
Gemma · S4MPL3BI4S
Params7.5B
Context128K
Q4_K_M5.3 GB
LL
Llm Jp 4 8b Instruct
Llm · mmnga-o
Params8.59B
Context64K
Q4_K_M5.3 GB
NE
Nemotron 3.5 Bolt E4B GSQ RCO
Nemotron · abenzerps
Params7.52B
Context128K
Smallest0.4 GB
CA
CAT Thinking 8B
Cat · mmnga-o
Params8.19B
Context40K
Q4_K_M5.0 GB
IN
InternLM2
InternLM · legraphista
Params7.74B
Context8K
Smallest1.9 GB