Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
PR
Prashna 7b
Prashna · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
MA
Mario Html v2 9B
Mario · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
TE
q-bert_terminis-7b
Terminis · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
Deepseek Coder 6.7b Base
DeepSeek · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
DA
DataPilot ArrowPro 7B RobinHood
Datapilot · mmnga
Params7.24B
Context4K
Q4_K_M4.4 GB
MO
Mox Tiny 1
Mox · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen3.8-Distilled-9B-NPU2
Qwen · Atomic-Germ
Params8.95B
Context256K
Smallest9.5 GB
FL
FLUX.2 klein 4B
Flux · adithya-balaji
Params3.88B
Context—
Smallest1.6 GB
MI
monology_openinstruct-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
QW
Qwen3 VL 4B Instruct Abliterated
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
QW
flux2 klein 4B uncensored text encoder
Qwen · Cordux
Params4B
Context—
Smallest2.4 GB
QW
Qwen3.5 4B Instruct
Qwen · openresearchtools
Params4.21B
Context256K
Q4_K_M2.7 GB
QV
QVikhr 3 4B Instruction
Qvikhr · Vikhrmodels
Params4B
Context40K
Q4_K_M2.5 GB
GE
Gemma SEA LION v4 4B VL
Gemma · aisingapore
Params3.88B
Context128K
Q4_K_M2.5 GB
ED
EdgeRock 4B
Edgerock · hkust-aaron
Params4.58B
Context256K
Q4_K_M2.9 GB
QW
Qwen2-7B-Instruct
Qwen · legraphista
Params7.62B
Context32K
Smallest1.9 GB
FL
Flan T5 Xxl
Flan · easygoing0114
Params4.76B
Context512
Q4_K_M2.9 GB
CL
Clef Flash
Clef · taurusduan
Params9.1B
Context256K
Q4_K_M6.0 GB
LL
llama 2 7B Guanaco QLoRA
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
QW
Qwen3.8 9B Distill Uncen Heretic
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
MI
Mixtral AI CyberTron Ultra
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
DeepSeek R1 Distill Qwen 8B Abliterated
DeepSeek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
TR
Trinity Nano Preview
Trinity · arcee-ai
Params6.12B
Context128K
Q4_K_M3.8 GB
OR
Orca 2 7B
Orca · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
RA
RakutenAI-7B-instruct
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
PH
Athene Phi 3.5 mini instruct orpo
Phi · bartowski
Params3.82B
Context128K
Q4_K_M2.3 GB
QW
DMind-3-mini
Qwen · Mungert
Params4B
Context256K
Q4_K_M2.5 GB
LI
Ling 3.0 Tiny Uncensored Abliterated
Ling · mradermacher
Params7.9B
Context128K
Q4_K_M4.8 GB
MI
MiMo Ornith 9B AGSI
Mimo · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB