Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
PR
Prashna 7b
Prashna · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
ReasoningTool use
MA
Mario Html v2 9B
Mario · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
ReasoningTool useVision
TE
q-bert_terminis-7b
Terminis · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
Deepseek Coder 6.7b Base
DeepSeek · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
DA
DataPilot ArrowPro 7B RobinHood
Datapilot · mmnga
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
MO
Mox Tiny 1
Mox · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
ReasoningRoleplay
QW
Qwen3.8-Distilled-9B-NPU2
Qwen · Atomic-Germ
Params8.95B
Context256K
Smallest9.5 GB
ReasoningTool useVision
FL
FLUX.2 klein 4B
Flux · adithya-balaji
Params3.88B
Context—
Smallest1.6 GB
GGUF
MI
monology_openinstruct-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
QW
Qwen3 VL 4B Instruct Abliterated
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
Vision
QW
flux2 klein 4B uncensored text encoder
Qwen · Cordux
Params4B
Context—
Smallest2.4 GB
Coding
QW
Qwen3.5 4B Instruct
Qwen · openresearchtools
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
QV
QVikhr 3 4B Instruction
Qvikhr · Vikhrmodels
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
GE
Gemma SEA LION v4 4B VL
Gemma · aisingapore
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
ED
EdgeRock 4B
Edgerock · hkust-aaron
Params4.58B
Context256K
Q4_K_M2.9 GB
Coding
QW
Qwen2-7B-Instruct
Qwen · legraphista
Params7.62B
Context32K
Smallest1.9 GB
GGUF
FL
Flan T5 Xxl
Flan · easygoing0114
Params4.76B
Context512
Q4_K_M2.9 GB
GGUF
CL
Clef Flash
Clef · taurusduan
Params9.1B
Context256K
Q4_K_M6.0 GB
CodingVision
LL
llama 2 7B Guanaco QLoRA
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.8 9B Distill Uncen Heretic
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
MI
Mixtral AI CyberTron Ultra
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
CodingReasoningRoleplay
DE
DeepSeek R1 Distill Qwen 8B Abliterated
DeepSeek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
Reasoning
TR
Trinity Nano Preview
Trinity · arcee-ai
Params6.12B
Context128K
Q4_K_M3.8 GB
GGUF
OR
Orca 2 7B
Orca · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
RA
RakutenAI-7B-instruct
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
PH
Athene Phi 3.5 mini instruct orpo
Phi · bartowski
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
QW
DMind-3-mini
Qwen · Mungert
Params4B
Context256K
Q4_K_M2.5 GB
Reasoning
LI
Ling 3.0 Tiny Uncensored Abliterated
Ling · mradermacher
Params7.9B
Context128K
Q4_K_M4.8 GB
Reasoning
MI
MiMo Ornith 9B AGSI
Mimo · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
ReasoningTool use

Other sizes

Keep browsing