Granite GGUF models

1547 GGUF models in the Granite family, quantized by 403 uploaders. Sizes run from 2M to 33.7B parameters. The longest trained context is 5120K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
GR
Granite 3.2 8b Instruct
Granite · ibm-research
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
Granite-3.3-2B-Instruct
Granite · unsloth
Params2.53B
Context128K
Q4_K_M1.5 GB
GGUF
GR
Granite Embedding 30m English
Granite · bartowski
Params30M
Context512
Q4_K_M0.0 GB
Embedding
GR
Granite Embedding 97m Multilingual R2
Granite · mykor
Params97M
Context32K
Q4_K_M0.1 GB
Embedding
GR
Granite Guardian 3.3 8b
Granite · mradermacher
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
GR
Granite 4.0 H Tiny Base
Granite · mradermacher
Params6.94B
Context1024K
Q4_K_M4.2 GB
GGUF
GR
Granite 4.0 350m Base
Granite · ibm-granite
Params352M
Context32K
Q4_K_M0.2 GB
GGUF
GR
ClinicalThought AI 8B
Granite · Raymond-dev-546730
Params8.17B
Context128K
Q4_K_M4.9 GB
ReasoningTool use
GR
granite vision 3.3 2b
Granite · ibm-granite
Params2.53B
Context128K
Q4_K_M1.5 GB
Tool useVision
GR
Granite 4.2 8b
Granite · mradermacher
Params8.79B
Context128K
Q4_K_M5.3 GB
ReasoningTool use
GR
IBM Granite 4.1 8b Heretic
Granite · mradermacher
Params8.38B
Context128K
Q4_K_M5.1 GB
GGUF
GR
granite 7b base
Granite · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GR
Granite 7b Instruct
Granite · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GR
Granite Speech 4.1 2b
Granite · ibm-granite
Params1.84B
Context4K
Q4_K_M1.1 GB
Vision
GR
Granite 4.1 Queen 30b
Granite · mradermacher
Params28.9B
Context128K
Q4_K_M17.5 GB
RoleplayTool useVision
GR
Granite Docling 258M
Granite · ibm-granite
Params164M
Context8K
Smallest0.3 GB
Vision
GR
Granite 4.0 H 350m
Granite · mrutkows
Params340M
Context1024K
Q4_K_M0.2 GB
Tool use
GR
Granite 4.1 8b
Granite · mradermacher
Params8.79B
Context128K
Q4_K_M5.3 GB
Tool use
GR
Granite 4.2 8B Uncensored
Granite · xCloudinfo
Params8.79B
Context128K
Q4_K_M5.3 GB
ReasoningTool use
GR
Granite 4.0 H Small
Granite · mradermacher
Params32.2B
Context1024K
Q4_K_M19.5 GB
GGUF
GR
Granite 3.3 2b Instruct Heretic Safety Defiltered
Granite · mradermacher
Params2.53B
Context128K
Q4_K_M1.5 GB
GGUF
GR
Granite 4.2 30B xVITA zhTW Uncensored
Granite · xCloudinfo
Params29.3B
Context128K
Q4_K_M17.7 GB
ReasoningTool use
GR
Granite 4.0 H Small BE
Granite · taronaeo
Params32.2B
Context1024K
Q4_K_M19.5 GB
GGUF
GR
Granite 4.0 Micro Base
Granite · ibm-granite
Params3.4B
Context128K
Q4_K_M2.1 GB
GGUF
GR
Granite Vision 3.3 2B BE
Granite · taronaeo
Params2.53B
Context128K
Q4_K_M1.5 GB
GGUF
GR
Granite Vision 3.2 2b
Granite · lmstudio-community
Params2.53B
Context16K
Q4_K_M1.5 GB
Tool useVision
GR
granite 34b code instruct
Granite · bartowski
Params33.7B
Context8K
Q4_K_M21.4 GB
CodingTool use
GR
Granite 3.0 1b A400M Instruct
Granite · mradermacher
Params1.33B
Context4K
Q4_K_M0.8 GB
Tool use
GR
Granite 3.2 8b Instruct
Granite · EasierAI
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
Granite 3.3 8b Instruct Apostate
Granite · mradermacher
Params8.37B
Context128K
Q4_K_M5.1 GB
ReasoningTool use

Keep browsing