Granite GGUF models

1547 GGUF models in the Granite family, quantized by 403 uploaders. Sizes run from 2M to 33.7B parameters. The longest trained context is 5120K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
GR
granite embedding reranker english r2 gguf q4 k m
Granite · keisuke-miyako
Params150M
Context8K
Q4_K_M0.1 GB
Embedding
GR
granite finetune
Granite · NagiEight
Params3.4B
Context128K
Q4_K_M2.1 GB
Tool use
GR
Granite 4.1 8b
Granite · viniciusianni
Params8.79B
Context128K
Smallest9.3 GB
GGUF
GR
Granite 3.0 3b A800M Base
Granite · davelsphere
Params3.37B
Context4K
Q4_K_M2.1 GB
GGUF
GR
Granite 3.1 1b ANAL DESTRUCTION
Granite · Novaciano
Params1.33B
Context128K
Smallest1.0 GB
RoleplayTool use
GR
granite embedding reranker english r2 gguf q8 0
Granite · keisuke-miyako
Params150M
Context8K
Smallest0.2 GB
Embedding
GR
Granite Embedding 30m English
Granite · r0x0d
Params30M
Context512
Q4_K_M0.0 GB
Embedding
GR
Granite 4.0 Micro Base
Granite · mrutkows
Params3.4B
Context128K
Q4_K_M2.1 GB
GGUF
GR
genie granite 2b Q4 K M v1
Granite · hanspat
Params2.53B
Context128K
Q4_K_M1.5 GB
ReasoningTool use
GR
granite 4.0 7B A1B Creative v0.2
Granite · WarlordHermes
Params6.67B
Context1024K
Q4_K_M4.0 GB
ReasoningTool use
GR
Granite Embedding 278m Multilingual
Granite · sushantag9
Params277M
Context512
Smallest0.3 GB
Embedding
GR
Granite Embedding Reranker English R2
Granite · jahn-idt
Params150M
Context8K
Smallest0.2 GB
Embedding
GR
Granite Guardian 3.2 5b Lora Harm Categories
Granite · nishparadox
Params17M
Context—
Smallest0.0 GB
GGUF
GR
granite4 h tiny q2k shards
Granite · Felipe0282829273
Params6.94B
Context1024K
Smallest2.6 GB
Tool use
GR
Granite 3.3 2b Instruct
Granite · EShawC
Params2.53B
Context128K
Q4_K_M1.5 GB
ReasoningTool use
GR
Granite 4.0 H 350m Base
Granite · aixk
Params340M
Context1024K
Q4_K_M0.2 GB
GGUF
GR
Granite 3.0 8b Instruct
Granite · jayakody2000lk
Params8.17B
Context4K
Smallest5.8 GB
Tool use
GR
granite 4.0 h micro gguf q8 0
Granite · keisuke-miyako
Params3.19B
Context1024K
Smallest3.4 GB
Tool use
GR
Granite Guardian 3.0 8b
Granite · davelsphere
Params8.17B
Context8K
Q4_K_M4.9 GB
GGUF
GR
Granite 4.1 3b Base
Granite · nitinrathi
Params3.4B
Context128K
Q4_K_M2.1 GB
GGUF
GR
granite finetune
Granite · tingytyyyy
Params3.19B
Context1024K
Smallest3.4 GB
Tool use
GR
RedHatAI Granite 3.1 8b Instruct
Granite · tensorblock
Params8.17B
Context128K
Smallest3.1 GB
Tool use
GR
instructLab granite 7b lab text to sql
Granite · ruslanmv
Params7.24B
Context32K
Smallest14.5 GB
GGUF
GR
pv granite
Granite · Giri424
Params8.79B
Context128K
Smallest9.3 GB
ReasoningTool use
GR
granite vision 3.2 2b
Granite · stefanberger
Params2.53B
Context128K
Q4_K_M1.5 GB
Tool useVision
GR
Granite 4.0 Micro
Granite · jagan546
Params3.4B
Context128K
Q4_K_M2.1 GB
GGUF
GR
RockMan256 Granite Docling 258M
Granite · RockMan256
Params164M
Context8K
Smallest0.3 GB
Vision
GR
Granite 3.3 2b Instruct Story Refiner
Granite · echos-keeper
Params2.53B
Context128K
Q4_K_M1.5 GB
GGUF
GR
Granite 4.0 350m
Granite · NikolayKozloff
Params352M
Context32K
Smallest0.4 GB
GGUF
GR
Granite 3.1 1b A400M Instruct
Granite · itlwas
Params1.33B
Context128K
Q4_K_M0.8 GB
Tool use

Keep browsing