Granite GGUF models

1547 GGUF models in the Granite family, quantized by 403 uploaders. Sizes run from 2M to 33.7B parameters. The longest trained context is 5120K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
GR
Granite 3.0 3b A800M Instruct
Granite · mradermacher
Params3.37B
Context4K
Q4_K_M2.1 GB
Tool use
GR
granite-3.1-8b-instruct
Granite · Mungert
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
Granite 3.3 8b Instruct Apostate
Granite · mradermacher
Params8.37B
Context128K
Q4_K_M5.1 GB
ReasoningTool use
GR
Granite 4.0 3b Vision
Granite · mrutkows
Params3.66B
Context128K
Q4_K_M2.2 GB
Tool useVision
GR
Granite 20b Code Instruct R1.1
Granite · mradermacher
Params20.1B
Context8K
Q4_K_M12.8 GB
CodingReasoningTool use
GR
Granite 3.0 2b Base
Granite · mradermacher
Params2.53B
Context4K
Q4_K_M1.5 GB
GGUF
GR
Granite 3.3 2b Instruct Literary Writer v0.1
Granite · mradermacher
Params2.53B
Context128K
Q4_K_M1.5 GB
GGUF
GR
Granite Speech 4.1 2b
Granite · mradermacher
Params1.84B
Context4K
Q4_K_M1.1 GB
GGUF
GR
Granite 3.3 2b Base
Granite · mradermacher
Params2.53B
Context128K
Q4_K_M1.5 GB
GGUF
GR
Granite 3.3 8b Instruct Abliterated
Granite · mradermacher
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
GR
Granite 3.1 Earthen v0.3 1B A400M
Granite · PJMixers-Dev
Params1.33B
Context128K
Q4_K_M0.8 GB
Tool use
GR
Half Granite Marvin
Granite · mradermacher
Params32.2B
Context1024K
Q4_K_M19.5 GB
Tool use
GR
Ibm Granite.Granite 7b Base
Granite · DevQuasar-8
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GR
Granite 4.0 Tiny Preview
Granite · mradermacher
Params6.67B
Context1024K
Q4_K_M4.0 GB
ReasoningTool use
GR
Granite 3.1 1b A400M Instruct
Granite · mradermacher
Params1.33B
Context128K
Q4_K_M0.8 GB
Tool use
GR
Granite 4.0 H Micro Base
Granite · mradermacher
Params3.19B
Context1024K
Q4_K_M1.9 GB
GGUF
GR
Granite 4.2 3b Heretic
Granite · tinyopsec
Params3.66B
Context128K
Q4_K_M2.2 GB
ReasoningTool use
GR
Granite Guardian 3.2 8b Factuality Detection
Granite · mradermacher
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
Iq Code Evmind v1 Granite 8b Instruct
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
Coding
GR
granite-guardian-3.2-5b
Granite · Mungert
Params5.78B
Context128K
Q4_K_M3.5 GB
Tool use
GR
Granite 4.1 3B Function Calling xLAM Unsloth
Granite · mradermacher
Params3.4B
Context128K
Q4_K_M2.1 GB
Tool use
GR
Granite Guardian 3.0 8b
Granite · RichardErkhov
Params8.17B
Context8K
Q4_K_M4.9 GB
GGUF
GR
Granite Embedding Small English R2
Granite · mradermacher
Params48M
Context8K
Q4_K_M0.0 GB
Embedding
GR
granite vision crispembed
Granite · cstr
Params—
Context—
Smallest0.0 GB
GGUF
GR
Ibm Granite.Granite 4.0 Tiny Base Preview
Granite · DevQuasar
Params6.67B
Context1024K
Q4_K_M4.1 GB
GGUF
GR
granite-3.0-1b-a400m-base
Granite · Mungert
Params1.38B
Context4K
Smallest0.6 GB
GGUF
GR
Granite 4.0 Tiny Preview
Granite · mradermacher
Params6.67B
Context1024K
Q4_K_M4.0 GB
ReasoningTool use
GR
Granite Embedding 107m Multilingual
Granite · lmstudio-community
Params107M
Context512
Q4_K_M0.1 GB
Embedding
GR
Granite 4.2 8b Heretic
Granite · mradermacher
Params8.79B
Context128K
Q4_K_M5.3 GB
ReasoningTool use
GR
Granite 3.2 8b Instruct Abliterated
Granite · mradermacher
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use

Keep browsing