Granite GGUF models

1547 GGUF models in the Granite family, quantized by 403 uploaders. Sizes run from 2M to 33.7B parameters. The longest trained context is 5120K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
GR
Granite 3.1 1b A400M Base
Granite · elichen-skymizer
Params1.33B
Context128K
Smallest1.4 GB
GGUF
GR
Granite Embedding 278m Multilingual
Granite · badger212
Params277M
Context512
Q4_K_M0.2 GB
Embedding
GR
Granite Embedding 30m English
Granite · stefanberger
Params30M
Context512
Q4_K_M0.0 GB
Embedding
GR
Granite Embedding 107m Multilingual
Granite · keisuke-miyako
Params107M
Context512
Q4_K_M0.1 GB
Embedding
GR
Granite Embedding 278m Multilingual
Granite · keisuke-miyako
Params277M
Context512
Smallest0.3 GB
Embedding
GR
granite-3b-code-instruct
Granite · Emilio407
Params3.48B
Context2K
Smallest3.7 GB
Coding
GR
Granite 4.0 Tiny Preview
Granite · AmpereComputing
Params6.67B
Context1024K
Smallest6.7 GB
ReasoningTool use
GR
Granite Guardian 3.2 5b
Granite · nishparadox
Params5.78B
Context128K
Smallest11.6 GB
Tool use
GR
Granite 3.1 8b Instruct
Granite · itlwas
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
Granite 8b Code Instruct 128k
Granite · sdalemorrey
Params8.05B
Context125K
Smallest8.6 GB
CodingTool use
GR
Granite Guardian 3.2 3b A800M
Granite · zero-puv
Params3.3B
Context128K
Smallest3.5 GB
Tool use
GR
ChaosNexus_Tuned_v1 GGUF
Granite · TunedChaos
Params8.79B
Context128K
Q4_K_M5.3 GB
CodingTool use
GR
granite embedding 125m english
Granite · stefanberger
Params124M
Context512
Q4_K_M0.1 GB
Embedding
GR
Granite 3.3 2b Instruct
Granite · sizzlebop
Params2.53B
Context128K
Q4_K_M1.5 GB
ReasoningTool use
GR
granite-8b-code-instruct
Granite · cobrakenji
Params8.05B
Context4K
Smallest3.9 GB
CodingTool use
GR
Granite 3.1 8b Instruct
Granite · Lucy-in-the-Sky
Params8.17B
Context128K
Smallest6.7 GB
Tool use
GR
Granite 3.3 2b Refiner
Granite · sillykiwi
Params2.53B
Context128K
Smallest2.1 GB
Vision
GR
GGUF.granite 4.0 Improved Enhanced Refined
Granite · mondk
Params1.63B
Context128K
Smallest1.2 GB
Tool use
GR
Granite 3.0 1b A400M Base
Granite · davelsphere
Params1.38B
Context4K
Q4_K_M0.9 GB
GGUF
GR
Granite 3.0 2b Instruct
Granite · mitulagr2
Params2.63B
Context4K
Smallest1.9 GB
Tool use
GR
Granite 3.1 2b Instruct
Granite · ObeJ
Params2.53B
Context128K
Smallest1.0 GB
Tool use
GR
Granite 3.3 2b Refiner
Granite · sillykiwi
Params2.53B
Context128K
Smallest2.7 GB
Vision
GR
Granite 3.3 2b Base
Granite · KushGupster
Params2.53B
Context128K
Q4_K_M1.5 GB
GGUF
GR
granite 4.0 350m unsloth bnb 4bit
Granite · apptonic
Params352M
Context32K
Smallest0.2 GB
Tool use
GR
granite-8b-code-base
Granite · YorkieOH10
Params8.05B
Context4K
Smallest8.6 GB
Coding
GR
IBM Granite Time Series TTM R2
Granite · JUSTiiNSANE
Params—
Context—
Smallest0.0 GB
GGUF
GR
Granite 3.0 8b Instruct
Granite · aashish1904
Params8.17B
Context4K
Smallest8.7 GB
Tool use
GR
Granite 4.2 30b
Granite · CompressedGemma
Params29.3B
Context128K
Smallest11.0 GB
ReasoningTool use
GR
granite 3.2 8b qiskit
Granite · Qiskit
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
Granite 4.1 8b Claude Opus 4.6 Thinking MAX
Granite · Varstes
Params8.79B
Context128K
Q4_K_M5.3 GB
Reasoning

Keep browsing