Granite GGUF models
1547 GGUF models in the Granite family, quantized by 403 uploaders. Sizes run from 2M to 33.7B parameters. The longest trained context is 5120K tokens.
GR
granite-7b-lab-xbcft
Granite · Samlxf
Params6.74B
Context2K
Smallest7.2 GB
GR
Granite Embedding 278m Multilingual
Granite · hongkeon
Params277M
Context512
Q4_K_M0.2 GB
GR
Granite 3.2 8b Instruct Preview
Granite · NikolayKozloff
Params8.17B
Context128K
Smallest8.7 GB
GR
Granite Flux Test
Granite · KushGupster
Params6.74B
Context4K
Smallest5.5 GB
GR
granite syllabusExtractor finetune
Granite · Ziloka
Params352M
Context32K
Q4_K_M0.2 GB
GR
granite speech 4.1 2b
Granite · vokra
Params2.42B
Context—
Smallest4.8 GB
GR
Granite 3.1 3b A800M Instruct
Granite · Felladrin
Params3.3B
Context128K
Smallest1.9 GB
GR
test instuctlab granite 7b lab ocpv doc
Granite · surote
Params6.74B
Context4K
Q4_K_M4.1 GB
GR
granite 4.0 micro gguf q8 0
Granite · keisuke-miyako
Params3.4B
Context128K
Smallest3.6 GB
GR
Granite Embedding English R2
Granite · mrutkows
Params149M
Context8K
Smallest0.2 GB
GR
Granite 4.0 H 1b
Granite · MdSajjadurRahaman
Params1.46B
Context1024K
Smallest2.9 GB
GR
MiniCPM3 4B
Granite · Felladrin
Params4.07B
Context32K
Smallest2.4 GB
GR
Granite Guardian 3.2 5b
Granite · mrutkows
Params5.78B
Context128K
Q4_K_M3.5 GB
GR
Granite Embedding 107m Multilingual
Granite · naver-ellm
Params107M
Context512
Q4_K_M0.1 GB
GR
granite 3.2 8b instruct heretic R5 K003 q8 0
Granite · kalle07
Params8.17B
Context128K
Smallest8.7 GB
GR
Granite 3.3 8b Instruct
Granite · cobrakenji
Params8.17B
Context128K
Q4_K_M4.9 GB
GR
Granite 4.0 H 350m
Granite · Felladrin
Params340M
Context1024K
Smallest0.2 GB
GR
Granite 3.2 8b Instruct
Granite · bchen-snp
Params8.17B
Context128K
Q4_K_M4.9 GB
GR
Granite 3.1 8b Instruct
Granite · cobrakenji
Params8.17B
Context128K
Q4_K_M4.9 GB
GR
instructLab granite 7b lab text to sql 4bits
Granite · ruslanmv
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
Granite 3.1 2b Base
Granite · itlwas
Params2.53B
Context128K
Q4_K_M1.5 GB
GR
Granite Flux Test
Granite · KushGupster
Params6.74B
Context4K
Q4_K_M4.1 GB
GR
granite guardian 3.2 3b a800m f16
Granite · u3man
Params3.3B
Context128K
Smallest6.6 GB
GR
Granite 3.0 8b Instruct R241014A
Granite · RedHatAI
Params8.17B
Context4K
Q4_K_M4.9 GB
GR
Granite 3.3b 2b Instruct
Granite · tanujrai
Params2.53B
Context128K
Smallest1.5 GB
GR
Granite 4.1 8b
Granite · mikelpr
Params8.79B
Context128K
Q4_K_M5.3 GB
GR
Granite 3.0 1b A400M Instruct
Granite · Felladrin
Params1.33B
Context4K
Smallest1.0 GB
GR
Granite 3.1 1b A400M Instruct
Granite · Felladrin
Params1.33B
Context128K
Smallest0.8 GB
GR
Granite 3.3 8b Instruct
Granite · pankaj217
Params8.17B
Context128K
Smallest8.7 GB
GR
granite 3.1 8b finetuned
Granite · raznaee
Params8.17B
Context128K
Smallest16.3 GB