GGUF models by kromquant
78 GGUF models quantized by kromquant on Hugging Face. Sizes run from 8.03B to 12B parameters. The longest trained context is 4192K tokens.
About kromquant on Hugging Face
kromquant has published 78 GGUF models that ggufmodels.com tracks, across families such as Testing, Siithamo and Spectre. The most downloaded are L3.1 Clouded Uchtave v0.1 8B GGUFs, L3.1 Aglow Vulca v0.1 8B GGUFs and Lacteous.
The quants kromquant publishes most often are Q8_0 (78), Q6_K (6), Q5_K_M (6) and Q4_K_M (6). Their Q4_K_M files range from 4.9 to 6.5 GB. Parameter counts run from 8.03B to 12B, and the longest trained context window is 4192K tokens.
Frequently asked questions
What GGUF models has kromquant published?
78 models are listed here, mainly from the Testing, Siithamo and Spectre families. The most downloaded is L3.1 Clouded Uchtave v0.1 8B GGUFs.
Which quants does kromquant publish?
The most common are Q8_0 (78 models), Q6_K (6 models), Q5_K_M (6 models) and Q4_K_M (6 models).