Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
CodeLlama 7b Instruct hf
Llama · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
DE
WhiteRabbitNeo V3 7B i1
Deephat · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
NE
models--automerger--NeuralsynthesisMultiverseex26-7B
Neuralsynthesis · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TH
The Trinity Coder 7B
The · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
MO
models--automerger--T3qT3qm7xp-7B
MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
nickynicky_mistral-7b-openorca-oasst_top1_2023-08-25-v2
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LE
LevelField
Levelfield · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
LL
Turkish Llama 8b Instruct v0.1
Llama · ytu-ce-cosmos
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3 8B Instruct Gradient 1048k GGUF smashed
Llama · PrunaAI
Params8B
Context1024K
Q4_K_M4.9 GB
GGUF
PH
Iceball Phi 3.5 mini instruct shareGPT v1.0 i1
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
QW
Git Commit Message Splitter Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
QW
Qwen Image 2.1 Uncensored
Qwen · 0Endzz
Params7.12B
Context—
Q4_K_M4.6 GB
GGUF
QW
Qwen3 DND Jan v1 256k Ctx Brainstorm40x 8B
Qwen · mradermacher
Params7.66B
Context256K
Q4_K_M4.7 GB
CodingReasoning
IC
IceEspressoRPv1 7b i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Replete LLM V2.5 Qwen 7b
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
GE
Fallen Gemma3 4B v1
Gemma · bartowski
Params3.88B
Context128K
Q4_K_M2.5 GB
GGUF
AG
Agents A1 4B
Agents · bartowski
Params4.21B
Context256K
Q4_K_M2.9 GB
Vision
JA
Jais-3-7B
Jais · IFM
Params9B
Context512K
Q4_K_M5.6 GB
ReasoningTool use
ST
Step3 VL 10B
Step · seanbailey518
Params8.19B
Context64K
Q4_K_M5.0 GB
Vision
QW
Qwen3.5-9B
Qwen · EcoAI-Lab
Params9.2B
Context256K
Smallest3.9 GB
GGUF
WE
WestLakev2 IreneRP Neural 7B slerp
Westlakev · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
VI
Video ORA 9B
Video · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
ReasoningVision
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
QW
typecastlm qwen3.5 3.8b
Qwen · mihailgribov
Params3.76B
Context256K
Smallest4.0 GB
GGUF
AP
Apertus SEA LION v4 8B IT
Apertus · aisingapore
Params8.05B
Context64K
Q4_K_M5.1 GB
ReasoningTool use
IR
IreneRP Neural 7B slerp
Irenerp · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
Roleplay
YI
Yi-1.5-9B
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
KA
KangalKhan RawRuby 7B
Kangalkhan · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MA
madlad400 7b mt
Madlad · google
Params8.3B
Context—
Smallest4.7 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
Coding

Other sizes

Keep browsing