Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
PA
pipizhao_pandalyst-7b-v1.1
Pandalyst · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
IN
InternVL3 8B Instruct
Internvl · ggml-org
Params7.61B
Context32K
Q4_K_M4.7 GB
Vision
TO
ToriiGate v0.4 7B
Toriigate · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Vision
LL
Llama 3.1 8B Stheno v3.4
Llama · Lewdiculous
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
MI
Mistral Pygmalion 7B
Mistral · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.5 9B Harmonic
Qwen · DJLougen
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoning
HO
K2 Horizon 3.7B Heretic
Horizon · mradermacher
Params5.06B
Context512K
Q4_K_M3.2 GB
GGUF
HO
K2 Horizon 7B Candor Uncensored
Horizon · candor-lab
Params9B
Context512K
Q4_K_M5.6 GB
ReasoningTool use
QW
Qwen3.8 9B Distill Uncen Heretic
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool useVision
MI
OpenHermes-2.5-Mistral-7B-x2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MI
MiniCPM V 2 6
MiniCPM · bartowski
Params7.6B
Context32K
Q4_K_M4.7 GB
CodingVision
QW
Qwythos 9B v2
Qwythos · mradermacher
Params9.2B
Context1024K
Q4_K_M5.8 GB
Reasoning
ID
ideogram4 turbo
Ideogram · ChrisColeTech
Params9.28B
Context256K
Q4_K_M5.0 GB
GGUF
MI
severian_anima-phi-neptune-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
Granite Guardian 4.1 8b
Granite · ibm-granite
Params8.38B
Context128K
Q4_K_M5.1 GB
ReasoningTool use
QW
PopiT Qwen3 4B Medical SFT 1128
Qwen · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
NE
NeoHorse 1 9 Uncensored
Neohorse · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
UN
ajibawa-2023_uncensored-jordan-7b
Uncensored · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · mradermacher
Params3.61B
Context128K
Q4_K_M2.2 GB
GGUF
GL
Huihui GLM 4.6V Flash abliterated
GLM · AliBilge
Params9.4B
Context128K
Q4_K_M6.2 GB
Vision
GE
gemma 4 E2B it UD japanese imatrix
Gemma · dahara1
Params4.65B
Context128K
Q4_K_M3.1 GB
Vision
QW
Qwen3-Vl-8B-Thinking-1M
Qwen · unsloth
Params8.19B
Context977K
Q4_K_M5.0 GB
ReasoningVision
GR
Granite 8b Code Instruct 4k
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
Coding
QW
WhereOPD Qwen3.5 4B
Qwen · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
GGUF
LL
Llama-3-Instruct-Neurona-8b-v2
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
CodingTool use
ST
L3-8B-Stheno-v3.2
Stheno · DavidAU
Params8B
Context8K
Q4_K_M4.9 GB
Roleplay
LL
nousresearch_yarn-llama-2-7b-64k
Llama · TheBloke
Params6.74B
Context64K
Q4_K_M4.1 GB
GGUF
QW
Qwen3 4B
Qwen · enacimie
Params4.02B
Context40K
Q4_K_M5.0 GB
GGUF
HO
Sakura K2 Horizon 7B
Horizon · webmp3
Params9B
Context512K
Smallest3.8 GB
GGUF
DE
Qwen2.5 MOE 2X1.5B DeepSeek Uncensored Censored 4B
DeepSeek · DavidAU
Params4.1B
Context128K
Q4_K_M2.5 GB
Reasoning

Other sizes

Keep browsing