Granite GGUF models

1547 GGUF models in the Granite family, quantized by 403 uploaders. Sizes run from 2M to 33.7B parameters. The longest trained context is 5120K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
GR
Granite Speech 4.1 2B Plus
Granite · memoravox
Params2.11B
Context—
Smallest1.7 GB
GGUF
GR
Granite 3.0 3b A800M Instruct
Granite · farpluto
Params3.37B
Context4K
Q4_K_M2.1 GB
Tool use
GR
Granite 3.0 1b A400M Base
Granite · itlwas
Params1.38B
Context4K
Q4_K_M0.9 GB
GGUF
GR
Granite 3.3 8b Instruct
Granite · atcodespaceexplore
Params8.17B
Context128K
Q4_K_M4.9 GB
ReasoningTool use
GR
Granite Estigia MultilanguageV3
Granite · franciscobdl
Params3.3B
Context128K
Smallest2.4 GB
Tool use
GR
granite docling 258M mine
Granite · nbdaaa
Params164M
Context8K
Smallest0.3 GB
Vision
GR
granite 4.0 350m por 16384
Granite · MAOZEDONG5567
Params266M
Context32K
Smallest0.5 GB
Tool use
GR
granite guardian 3.2 3b a800m Q4 K M
Granite · u3man
Params3.3B
Context128K
Q4_K_M2.0 GB
Tool use
GR
Granite 3.0 2b Instruct
Granite · psolarvi
Params2.63B
Context4K
Q4_K_M1.6 GB
Tool use
GR
granite 4 h 3b unsloth fft
Granite · sjpritchard
Params3.19B
Context1024K
Q4_K_M1.9 GB
Tool use
GR
guard granite
Granite · faradailab
Params2.53B
Context8K
Q4_K_M1.5 GB
GGUF
GR
Granite Speech 4.1 2B NAR
Granite · memoravox
Params2.25B
Context—
Smallest1.8 GB
GGUF
GR
Granite 3.1 3b A800M Instruct
Granite · fernandoruiz
Params3.3B
Context128K
Smallest1.9 GB
Tool use
GR
YouLearn Granite4 H1B V11 FT564 Merged F16
Granite · Tabish244
Params1.46B
Context1024K
Q4_K_M0.9 GB
Tool use
GR
Granite 3 Flux 1
Granite · KushGupster
Params8.37B
Context128K
Smallest8.9 GB
Tool use
GR
granite embedding 278m shindy multilingual
Granite · shindy-dev
Params277M
Context512
Q4_K_M0.2 GB
Embedding
GR
Granite 3.0 2b Base
Granite · davelsphere
Params2.53B
Context4K
Q4_K_M1.5 GB
GGUF
GR
Granite 3.3 8b Instruct
Granite · lawrencenull
Params8.17B
Context128K
Smallest3.1 GB
ReasoningTool use
GR
Granite 3.3 8b Instruct
Granite · AmpereComputing
Params8.17B
Context128K
Smallest8.2 GB
ReasoningTool use
GR
Granite 4.1 30b
Granite · viniciusianni
Params28.9B
Context128K
Smallest30.7 GB
GGUF
GR
granite 4.1 3b heretic
Granite · realoperator42
Params3.4B
Context128K
Smallest2.0 GB
Tool use
GR
Granite 3.0 8b Instruct
Granite · 1FriendlyDoge
Params8.17B
Context4K
Smallest8.7 GB
Tool use
GR
granite 3.0 8b instruct Q4 K M
Granite · eformat
Params8.17B
Context4K
Q4_K_M4.9 GB
Tool use
GR
Granite 3.1 8b Base
Granite · itlwas
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
GR
Granite 4.0 Micro
Granite · Edge-Quant
Params3.4B
Context128K
Q4_K_M2.1 GB
GGUF
GR
granite embedding small english r2 gguf q4 k m
Granite · keisuke-miyako
Params149M
Context8K
Q4_K_M0.1 GB
Embedding
GR
Granite 3.2 8b Instruct
Granite · doaonduty
Params8.17B
Context128K
Smallest8.7 GB
Tool use
GR
Granite 3.3 2b Instruct
Granite · sizzlebop
Params2.53B
Context128K
Smallest2.7 GB
ReasoningTool use
GR
Granite Embedding
Granite · rafpigna
Params312M
Context32K
Smallest0.3 GB
Embedding
GR
Granite Embedding 107m Multilingual
Granite · mrutkows
Params107M
Context512
Q4_K_M0.1 GB
Embedding

Keep browsing