Granite GGUF models

1547 GGUF models in the Granite family, quantized by 403 uploaders. Sizes run from 2M to 33.7B parameters. The longest trained context is 5120K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
GR
Ibm Granite Granite 3.0 2b Base
Granite · tensorblock
Params2.53B
Context4K
Smallest1.0 GB
GGUF
GR
granite embedding 97m multilingual r2
Granite · cstr
Params102M
Context—
Smallest0.1 GB
GGUF
GR
Huihui Ai.Granite 3.1 8b Instruct Abliterated
Granite · DevQuasar-5
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
Granite 3.0 2b Instruct
Granite · mrutkows
Params2.63B
Context4K
Q4_K_M1.6 GB
Tool use
GR
Bloomee v1 Clm Nasasmd Granite4.1 3b
Granite · mradermacher
Params3.4B
Context128K
Q4_K_M2.1 GB
Tool use
GR
Models Huihui Ai Granite 3.2 8b Instruct Abliterated
Granite · MaziyarPanahi
Params8.17B
Context128K
Smallest5.6 GB
Tool use
GR
Granite Speech 5.0 470M TurboCTC
Granite · 17slever17
Params473M
Context—
Smallest0.5 GB
GGUF
GR
Granite 20b Code Base R1.1
Granite · mradermacher
Params20.1B
Context8K
Q4_K_M12.8 GB
CodingReasoning
GR
Granite 4.0 Micro
Granite · mradermacher
Params3.4B
Context128K
Q4_K_M2.1 GB
Tool use
GR
Granite 4.2 30B Fable Distill
Granite · mradermacher
Params29.3B
Context128K
Q4_K_M17.7 GB
ReasoningTool use
GR
Granite 4.2 3b Heretic Abliterated
Granite · mradermacher
Params3.66B
Context128K
Q4_K_M2.2 GB
ReasoningTool use
GR
Granite 3.1 8b Instruct Abliterated
Granite · mradermacher
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
granite embedding 107m multilingual
Granite · cstr
Params107M
Context514
Smallest0.1 GB
Embedding
GR
Granite 3.3 2B Avg
Granite · mradermacher
Params2.53B
Context128K
Q4_K_M1.5 GB
Tool use
GR
Granite 3.3 2B Avg SliceWeighted
Granite · mradermacher
Params2.53B
Context128K
Q4_K_M1.5 GB
Tool use
GR
Ibm Granite.Granite 4.0 H 350m
Granite · DevQuasar
Params340M
Context1024K
Q4_K_M0.2 GB
Tool use
GR
granite-4.2-30b
Granite · kingjones777
Params29.3B
Context128K
Smallest30.7 GB
ReasoningTool use
GR
Granite Speech 3.3 8b
Granite · mradermacher
Params8.17B
Context128K
Q4_K_M4.9 GB
ReasoningTool useVision
GR
Granite 4.1 3B SFT Claude Opus Reasoning Unsloth
Granite · mradermacher
Params3.4B
Context128K
Q4_K_M2.1 GB
ReasoningTool use
GR
Ibm Granite.Granite 3.0 2b Base
Granite · DevQuasar-8
Params2.53B
Context4K
Q4_K_M1.5 GB
GGUF
GR
granite embedding 311m multilingual r2
Granite · cstr
Params323M
Context—
Smallest0.3 GB
GGUF
GR
Granite 4.1 3b Abilerated Unc
Granite · mradermacher
Params3.4B
Context128K
Q4_K_M2.1 GB
Tool use
GR
granite speech 4.1 2b
Granite · scholzmx
Params2.39B
Context—
Smallest1.5 GB
GGUF
GR
Granite Embedding 311m Multilingual R2
Granite · SandLogicTechnologies
Params312M
Context8K
Smallest0.2 GB
CodingEmbedding
GR
Granite Guardian 3.0 8b
Granite · mradermacher
Params8.17B
Context8K
Q4_K_M4.9 GB
GGUF
GR
Ibm Granite.Granite 3.0 1b A400M Base
Granite · DevQuasar-8
Params1.38B
Context4K
Q4_K_M0.9 GB
GGUF
GR
Granite 4.1 8B SFT Claude Opus Reasoning Unsloth
Granite · ermiaazarkhalili
Params8.79B
Context128K
Q4_K_M5.3 GB
ReasoningTool use
GR
granite-3b-code-instruct
Granite · ibm-granite
Params3.48B
Context2K
Q4_K_M2.1 GB
CodingTool use
GR
granite-4.2-3b
Granite · vultr
Params3.66B
Context128K
Q4_K_M2.5 GB
ReasoningTool use
GR
Granite 4.0 Micro Heretic Uncensored
Granite · mradermacher
Params3.4B
Context128K
Q4_K_M2.1 GB
Tool use

Keep browsing