GGUF models by mrutkows

63 GGUF models quantized by mrutkows on Hugging Face. Sizes run from 30M to 117B parameters. The longest trained context is 1024K tokens.

Most downloadedNewestLargestLongest contextSmallest
GR
Granite Docling 258M
Granite · mrutkows
Params164M
Context8K
Q4_K_M0.1 GB
Vision
GR
Granite 4.0 Tiny Base Preview
Granite · mrutkows
Params6.67B
Context1024K
Q4_K_M4.1 GB
GGUF
GR
Granite 4.0 350m Base
Granite · mrutkows
Params352M
Context32K
Q4_K_M0.2 GB
GGUF
GR
Granite 3.1 3b A800M Instruct
Granite · mrutkows
Params3.3B
Context128K
Q4_K_M2.0 GB
Tool use
GR
Granite 3.0 2b Base
Granite · mrutkows
Params2.53B
Context4K
Q4_K_M1.5 GB
GGUF
GR
Granite 3.1 1b A400M Base
Granite · mrutkows
Params1.33B
Context128K
Q4_K_M0.8 GB
GGUF
GR
Granite 4.0 H 350m Base
Granite · mrutkows
Params340M
Context1024K
Q4_K_M0.2 GB
GGUF
GR
granite vision 3.3 2b
Granite · mrutkows
Params2.53B
Context128K
Q4_K_M1.5 GB
Tool useVision
GR
granite vision 3.2 2b
Granite · mrutkows
Params2.53B
Context128K
Q4_K_M1.5 GB
Tool useVision
GR
Granite 3.1 3b A800M Base
Granite · mrutkows
Params3.3B
Context128K
Q4_K_M2.0 GB
GGUF
GP
Gpt Oss 120b
gpt-oss · mrutkows
Params117B
Context128K
Q4_K_M87.9 GB
Tool use
GR
Granite 3.1 2b Instruct
Granite · mrutkows
Params2.53B
Context128K
Q4_K_M1.5 GB
Tool use
GR
Granite Speech 4.1 2b
Granite · mrutkows
Params1.84B
Context4K
Q4_K_M1.1 GB
Vision
GR
Granite 4.0 H Micro Base
Granite · mrutkows
Params3.19B
Context1024K
Q4_K_M1.9 GB
GGUF
GR
Granite Guardian 3.1 8b
Granite · mrutkows
Params8.17B
Context128K
Q4_K_M4.9 GB
Tool use
GR
Granite Guardian 3.0 8b
Granite · mrutkows
Params8.17B
Context8K
Q4_K_M4.9 GB
GGUF
GR
Granite 3.0 3b A800M Instruct
Granite · mrutkows
Params3.37B
Context4K
Q4_K_M2.1 GB
Tool use
GR
Granite 3.0 3b A800M Base
Granite · mrutkows
Params3.37B
Context4K
Q4_K_M2.1 GB
GGUF
GR
granite vision 3.3 2b chart2csv preview
Granite · mrutkows
Params2.53B
Context128K
Q4_K_M1.5 GB
Tool useVision
GR
Granite Embedding 30m English
Granite · mrutkows
Params30M
Context512
Q4_K_M0.0 GB
Embedding
GR
Granite 4.1 30b
Granite · mrutkows
Params28.9B
Context128K
Q4_K_M17.5 GB
Tool use
GR
Granite Embedding Reranker English R2
Granite · mrutkows
Params150M
Context8K
Smallest0.2 GB
Embedding
GR
Granite Embedding 278m Multilingual
Granite · mrutkows
Params277M
Context512
Q4_K_M0.2 GB
Embedding
GR
Granite 4.0 H Tiny
Granite · mrutkows
Params6.94B
Context1024K
Q4_K_M4.2 GB
Tool use
GR
Granite Guardian 3.2 3b A800M
Granite · mrutkows
Params3.3B
Context128K
Q4_K_M2.0 GB
Tool use
GR
Granite 4.0 Micro Base
Granite · mrutkows
Params3.4B
Context128K
Q4_K_M2.1 GB
GGUF
GR
Granite Guardian 3.0 2b
Granite · mrutkows
Params2.53B
Context8K
Q4_K_M1.5 GB
GGUF
GR
Granite 4.0 H Tiny Base
Granite · mrutkows
Params6.94B
Context1024K
Q4_K_M4.2 GB
GGUF
GR
granite embedding 125m english
Granite · mrutkows
Params124M
Context512
Q4_K_M0.1 GB
Embedding
GR
Granite Embedding Small English R2
Granite · mrutkows
Params48M
Context8K
Smallest0.1 GB
Embedding

Keep browsing