GGUF models by mrutkows
63 GGUF models quantized by mrutkows on Hugging Face. Sizes run from 30M to 117B parameters. The longest trained context is 1024K tokens.
GR
Granite Docling 258M
Granite · mrutkows
Params164M
Context8K
Q4_K_M0.1 GB
GR
Granite 4.0 Tiny Base Preview
Granite · mrutkows
Params6.67B
Context1024K
Q4_K_M4.1 GB
GR
Granite 4.0 350m Base
Granite · mrutkows
Params352M
Context32K
Q4_K_M0.2 GB
GR
Granite 3.1 3b A800M Instruct
Granite · mrutkows
Params3.3B
Context128K
Q4_K_M2.0 GB
GR
Granite 3.0 2b Base
Granite · mrutkows
Params2.53B
Context4K
Q4_K_M1.5 GB
GR
Granite 3.1 1b A400M Base
Granite · mrutkows
Params1.33B
Context128K
Q4_K_M0.8 GB
GR
Granite 4.0 H 350m Base
Granite · mrutkows
Params340M
Context1024K
Q4_K_M0.2 GB
GR
granite vision 3.3 2b
Granite · mrutkows
Params2.53B
Context128K
Q4_K_M1.5 GB
GR
granite vision 3.2 2b
Granite · mrutkows
Params2.53B
Context128K
Q4_K_M1.5 GB
GR
Granite 3.1 3b A800M Base
Granite · mrutkows
Params3.3B
Context128K
Q4_K_M2.0 GB
GP
Gpt Oss 120b
gpt-oss · mrutkows
Params117B
Context128K
Q4_K_M87.9 GB
GR
Granite 3.1 2b Instruct
Granite · mrutkows
Params2.53B
Context128K
Q4_K_M1.5 GB
GR
Granite Speech 4.1 2b
Granite · mrutkows
Params1.84B
Context4K
Q4_K_M1.1 GB
GR
Granite 4.0 H Micro Base
Granite · mrutkows
Params3.19B
Context1024K
Q4_K_M1.9 GB
GR
Granite Guardian 3.1 8b
Granite · mrutkows
Params8.17B
Context128K
Q4_K_M4.9 GB
GR
Granite Guardian 3.0 8b
Granite · mrutkows
Params8.17B
Context8K
Q4_K_M4.9 GB
GR
Granite 3.0 3b A800M Instruct
Granite · mrutkows
Params3.37B
Context4K
Q4_K_M2.1 GB
GR
Granite 3.0 3b A800M Base
Granite · mrutkows
Params3.37B
Context4K
Q4_K_M2.1 GB
GR
granite vision 3.3 2b chart2csv preview
Granite · mrutkows
Params2.53B
Context128K
Q4_K_M1.5 GB
GR
Granite Embedding 30m English
Granite · mrutkows
Params30M
Context512
Q4_K_M0.0 GB
GR
Granite 4.1 30b
Granite · mrutkows
Params28.9B
Context128K
Q4_K_M17.5 GB
GR
Granite Embedding Reranker English R2
Granite · mrutkows
Params150M
Context8K
Smallest0.2 GB
GR
Granite Embedding 278m Multilingual
Granite · mrutkows
Params277M
Context512
Q4_K_M0.2 GB
GR
Granite 4.0 H Tiny
Granite · mrutkows
Params6.94B
Context1024K
Q4_K_M4.2 GB
GR
Granite Guardian 3.2 3b A800M
Granite · mrutkows
Params3.3B
Context128K
Q4_K_M2.0 GB
GR
Granite 4.0 Micro Base
Granite · mrutkows
Params3.4B
Context128K
Q4_K_M2.1 GB
GR
Granite Guardian 3.0 2b
Granite · mrutkows
Params2.53B
Context8K
Q4_K_M1.5 GB
GR
Granite 4.0 H Tiny Base
Granite · mrutkows
Params6.94B
Context1024K
Q4_K_M4.2 GB
GR
granite embedding 125m english
Granite · mrutkows
Params124M
Context512
Q4_K_M0.1 GB
GR
Granite Embedding Small English R2
Granite · mrutkows
Params48M
Context8K
Smallest0.1 GB