Granite GGUF models
1547 GGUF models in the Granite family, quantized by 403 uploaders. Sizes run from 2M to 33.7B parameters. The longest trained context is 5120K tokens.
GR
Granite 3.2 8b Instruct
Granite · ibm-research
Params8.17B
Context128K
Q4_K_M4.9 GB
GR
Granite-3.3-2B-Instruct
Granite · unsloth
Params2.53B
Context128K
Q4_K_M1.5 GB
GR
Granite Embedding 30m English
Granite · bartowski
Params30M
Context512
Q4_K_M0.0 GB
GR
Granite Embedding 97m Multilingual R2
Granite · mykor
Params97M
Context32K
Q4_K_M0.1 GB
GR
Granite Guardian 3.3 8b
Granite · mradermacher
Params8.17B
Context128K
Q4_K_M4.9 GB
GR
Granite 4.0 H Tiny Base
Granite · mradermacher
Params6.94B
Context1024K
Q4_K_M4.2 GB
GR
Granite 4.0 350m Base
Granite · ibm-granite
Params352M
Context32K
Q4_K_M0.2 GB
GR
ClinicalThought AI 8B
Granite · Raymond-dev-546730
Params8.17B
Context128K
Q4_K_M4.9 GB
GR
granite vision 3.3 2b
Granite · ibm-granite
Params2.53B
Context128K
Q4_K_M1.5 GB
GR
Granite 4.2 8b
Granite · mradermacher
Params8.79B
Context128K
Q4_K_M5.3 GB
GR
IBM Granite 4.1 8b Heretic
Granite · mradermacher
Params8.38B
Context128K
Q4_K_M5.1 GB
GR
granite 7b base
Granite · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB
GR
Granite 7b Instruct
Granite · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GR
Granite Speech 4.1 2b
Granite · ibm-granite
Params1.84B
Context4K
Q4_K_M1.1 GB
GR
Granite 4.1 Queen 30b
Granite · mradermacher
Params28.9B
Context128K
Q4_K_M17.5 GB
GR
Granite Docling 258M
Granite · ibm-granite
Params164M
Context8K
Smallest0.3 GB
GR
Granite 4.0 H 350m
Granite · mrutkows
Params340M
Context1024K
Q4_K_M0.2 GB
GR
Granite 4.1 8b
Granite · mradermacher
Params8.79B
Context128K
Q4_K_M5.3 GB
GR
Granite 4.2 8B Uncensored
Granite · xCloudinfo
Params8.79B
Context128K
Q4_K_M5.3 GB
GR
Granite 4.0 H Small
Granite · mradermacher
Params32.2B
Context1024K
Q4_K_M19.5 GB
GR
Granite 3.3 2b Instruct Heretic Safety Defiltered
Granite · mradermacher
Params2.53B
Context128K
Q4_K_M1.5 GB
GR
Granite 4.2 30B xVITA zhTW Uncensored
Granite · xCloudinfo
Params29.3B
Context128K
Q4_K_M17.7 GB
GR
Granite 4.0 H Small BE
Granite · taronaeo
Params32.2B
Context1024K
Q4_K_M19.5 GB
GR
Granite 4.0 Micro Base
Granite · ibm-granite
Params3.4B
Context128K
Q4_K_M2.1 GB
GR
Granite Vision 3.3 2B BE
Granite · taronaeo
Params2.53B
Context128K
Q4_K_M1.5 GB
GR
Granite Vision 3.2 2b
Granite · lmstudio-community
Params2.53B
Context16K
Q4_K_M1.5 GB
GR
granite 34b code instruct
Granite · bartowski
Params33.7B
Context8K
Q4_K_M21.4 GB
GR
Granite 3.0 1b A400M Instruct
Granite · mradermacher
Params1.33B
Context4K
Q4_K_M0.8 GB
GR
Granite 3.2 8b Instruct
Granite · EasierAI
Params8.17B
Context128K
Q4_K_M4.9 GB
GR
Granite 3.3 8b Instruct Apostate
Granite · mradermacher
Params8.37B
Context128K
Q4_K_M5.1 GB