Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
GE
Gemma 4 E4B It Erotica
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
GGUF
SC
ajibawa-2023_scarlett-7b
Scarlett · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
DE
Llama 3.1 TAIDE R1 8B Chat
DeepSeek · voidful
Params8.52B
Context128K
Smallest17.1 GB
Reasoning
LL
Llama 3.1 Storm 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Reasoning
QW
Qwen3.5 9B Artemis
Qwen · turintech
Params8.95B
Context256K
Q4_K_M5.7 GB
ReasoningTool useVision
LL
LLaDA 1.5
Llada · mradermacher
Params8.02B
Context4K
Q4_K_M4.9 GB
GGUF
GE
gemma 3n E2B it
Gemma · ggml-org
Params4.46B
Context32K
Smallest4.8 GB
GGUF
JA
Javi0.3 4B
Javi · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
ReasoningTool use
AI
airoboros l2 7B 2.2.1
Airoboros · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GR
Granite 3.2 8b Instruct
Granite · lmstudio-community
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen3 Psychological Reasoning 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
Reasoning
AT
Athena-CodeZephyr-7B-v0.2
Athena · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
DE
deepseek coder 6.7B kexer
DeepSeek · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
Hermes-2-Pro-Mistral-7B-infinity
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
QW
Qwimi 4B
Qwimi · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
Tool use
TI
Tifa DeepsexV2 7b MGRPO GGUF Q8
Tifa · ValueFX9507
Params7.6B
Context32K
Smallest8.1 GB
Roleplay
GR
Granite Switch 4.1 8b
Granite · ibm-granite
Params8.97B
Context128K
Q4_K_M6.4 GB
Tool use
HE
HERETICSEEK 7B Ditill
Hereticseek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
PI
maywell_pivot-0.1-early
Pivot · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TH
TherapyLlama 8B v1 i1
Therapyllama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
MiniAI Quata2 4b
Miniai · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
Reasoning
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
Reasoning
ST
starcoder2 7b
StarCoder · QuantFactory
Params7.2B
Context16K
Q4_K_M4.5 GB
Coding
PP
pplx decider flash v1.2
Pplx · Jakevin
Params4.84B
Context256K
Q4_K_M3.1 GB
Vision
TE
Teutonic I Reliquary RL
Teutonic · bartowski
Params8.95B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
QW
Meissa Qwen2.5 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
Roleplay
LL
Infinity Instruct 7M Gen Llama3_1 8B
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GE
Gemma 4 E2B It Ultra Uncensored Heretic
Gemma · princepppp
Params4.65B
Context128K
Q4_K_M3.4 GB
Vision
ST
L3-8B-Stheno-v3.3-32K
Stheno · Lewdiculous
Params8B
Context32K
Q4_K_M4.9 GB
Roleplay
GR
Granite 8b Code Instruct 4k
Granite · mradermacher
Params8.05B
Context4K
Q4_K_M4.9 GB
CodingTool use

Other sizes

Keep browsing