Granite GGUF models

1547 GGUF models in the Granite family, quantized by 403 uploaders. Sizes run from 2M to 33.7B parameters. The longest trained context is 5120K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
GR
granite 2B
Granite · SRA25
Params2.53B
Context128K
Smallest1.5 GB
ReasoningTool use
GR
granite guardian hap 125m f16
Granite · u3man
Params125M
Context512
Smallest0.3 GB
GGUF
GR
granite 7b lab zb1
Granite · refarde
Params6.74B
Context4K
Smallest13.5 GB
GGUF
GR
granite 8b code BE
Granite · prajwal-ibm
Params8.05B
Context125K
Smallest16.1 GB
GGUF
GR
Granite Flux Test
Granite · KushGupster
Params6.74B
Context4K
Smallest3.8 GB
GGUF
GR
Granite 3.3 2b Instruct
Granite · gzitei
Params2.53B
Context128K
Smallest1.4 GB
ReasoningTool use
GR
granite 7b lab Q4 K M
Granite · dgpmakes
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
GR
granite 7b lab martius2
Granite · Matteoleme
Params6.74B
Context4K
Smallest13.5 GB
GGUF
GR
Granite 3.3 2b Instruct
Granite · neoservicios
Params2.53B
Context128K
Q4_K_M1.5 GB
ReasoningTool use
GR
Granite 3.0 3b A800M Instruct
Granite · Felladrin
Params3.37B
Context4K
Smallest2.5 GB
Tool use
GR
granite 7b lab martius1
Granite · Matteoleme
Params6.74B
Context4K
Smallest13.5 GB
GGUF
GR
Granite Guardian
Granite · jbarguti
Params2.53B
Context128K
Smallest2.7 GB
Tool use
GR
Granite 3.2 8b Instruct
Granite · jbarguti
Params8.17B
Context128K
Smallest8.7 GB
Tool use
GR
Granite 3.2 2b Instruct
Granite · neoservicios
Params2.53B
Context128K
Q4_K_M1.5 GB
Tool use
GR
ilab trained granite 7b lab
Granite · alinahid477
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GR
granite 3.2 2b instruct BE fp16
Granite · prajwal-ibm
Params2.53B
Context128K
Smallest5.1 GB
GGUF
GR
medical granite 350m Q8
Granite · sandepaAI
Params352M
Context32K
Smallest0.4 GB
Tool use
GR
Estigia granite q5
Granite · franciscobdl
Params6.94B
Context1024K
Smallest4.9 GB
Tool use
GR
instructlab granite 7b lab policy v2 Q4 K M
Granite · DeanChanterIBM
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GR
instructlab granite 7b lab trained
Granite · Suranjan
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GR
bolt embedding small
Granite · aisquared
Params48M
Context8K
Smallest0.1 GB
Embedding
GR
bolt embedding large
Granite · aisquared
Params149M
Context8K
Smallest0.3 GB
Embedding
GR
Granite4T kiri
Granite · mikrisphere
Params6.94B
Context1024K
Q4_K_M4.2 GB
Tool use
GR
granite 4.1 8b banking input guardrails
Granite · sksanjoo23
Params50M
Context—
Smallest0.2 GB
GGUF
GR
Granite 3.1 3B FR
Granite · jquessada
Params3.3B
Context128K
Smallest3.5 GB
Tool use
GR
Granite Speech 5.0 470M TurboCTC
Granite · lipik
Params473M
Context—
Smallest0.5 GB
GGUF
GR
Granite Speech 4.1 2B
Granite · lipik
Params2.31B
Context—
Smallest2.6 GB
GGUF

Keep browsing