Granite GGUF models

1547 GGUF models in the Granite family, quantized by 403 uploaders. Sizes run from 2M to 33.7B parameters. The longest trained context is 5120K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
GR
granite-7b-lab-xbcft
Granite · Samlxf
Params6.74B
Context2K
Smallest7.2 GB
GGUF
GR
Granite Embedding 278m Multilingual
Granite · hongkeon
Params277M
Context512
Q4_K_M0.2 GB
Embedding
GR
Granite 3.2 8b Instruct Preview
Granite · NikolayKozloff
Params8.17B
Context128K
Smallest8.7 GB
Tool use
GR
Granite Flux Test
Granite · KushGupster
Params6.74B
Context4K
Smallest5.5 GB
GGUF
GR
granite syllabusExtractor finetune
Granite · Ziloka
Params352M
Context32K
Q4_K_M0.2 GB
Tool use
GR
granite speech 4.1 2b
Granite · vokra
Params2.42B
Context—
Smallest4.8 GB
GGUF
GR
Granite 3.1 3b A800M Instruct
Granite · Felladrin
Params3.3B
Context128K
Smallest1.9 GB
Tool use
GR
test instuctlab granite 7b lab ocpv doc
Granite · surote
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GR
granite 4.0 micro gguf q8 0
Granite · keisuke-miyako
Params3.4B
Context128K
Smallest3.6 GB
Tool use
GR
Granite Embedding English R2
Granite · mrutkows
Params149M
Context8K
Smallest0.2 GB
Embedding
GR
Granite 4.0 H 1b
Granite · MdSajjadurRahaman
Params1.46B
Context1024K
Smallest2.9 GB
Tool use
GR
MiniCPM3 4B
Granite · Felladrin
Params4.07B
Context32K
Smallest2.4 GB
Tool use
GR
Granite Guardian 3.2 5b
Granite · mrutkows
Params5.78B
Context128K
Q4_K_M3.5 GB
Tool use
GR
Granite Embedding 107m Multilingual
Granite · naver-ellm
Params107M
Context512
Q4_K_M0.1 GB
Embedding
GR
granite 3.2 8b instruct heretic R5 K003 q8 0
Granite · kalle07
Params8.17B
Context128K
Smallest8.7 GB
Tool use
GR
Granite 3.3 8b Instruct
Granite · cobrakenji
Params8.17B
Context128K
Q4_K_M4.9 GB
ReasoningTool use
GR
Granite 4.0 H 350m
Granite · Felladrin
Params340M
Context1024K
Smallest0.2 GB
Tool use
GR
Granite 3.2 8b Instruct
Granite · bchen-snp
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
Granite 3.1 8b Instruct
Granite · cobrakenji
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
instructLab granite 7b lab text to sql 4bits
Granite · ruslanmv
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
Granite 3.1 2b Base
Granite · itlwas
Params2.53B
Context128K
Q4_K_M1.5 GB
GGUF
GR
Granite Flux Test
Granite · KushGupster
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GR
granite guardian 3.2 3b a800m f16
Granite · u3man
Params3.3B
Context128K
Smallest6.6 GB
Tool use
GR
Granite 3.0 8b Instruct R241014A
Granite · RedHatAI
Params8.17B
Context4K
Q4_K_M4.9 GB
Tool use
GR
Granite 3.3b 2b Instruct
Granite · tanujrai
Params2.53B
Context128K
Smallest1.5 GB
ReasoningTool use
GR
Granite 4.1 8b
Granite · mikelpr
Params8.79B
Context128K
Q4_K_M5.3 GB
GGUF
GR
Granite 3.0 1b A400M Instruct
Granite · Felladrin
Params1.33B
Context4K
Smallest1.0 GB
Tool use
GR
Granite 3.1 1b A400M Instruct
Granite · Felladrin
Params1.33B
Context128K
Smallest0.8 GB
Tool use
GR
Granite 3.3 8b Instruct
Granite · pankaj217
Params8.17B
Context128K
Smallest8.7 GB
ReasoningTool use
GR
granite 3.1 8b finetuned
Granite · raznaee
Params8.17B
Context128K
Smallest16.3 GB
Tool use

Keep browsing