GGUF models by featherless-ai-quants
3092 GGUF models quantized by featherless-ai-quants on Hugging Face. Sizes run from 494M to 72.7B parameters. The longest trained context is 1024K tokens.
About featherless-ai-quants on Hugging Face
featherless-ai-quants has published 3,092 GGUF models that ggufmodels.com tracks, across families such as Llama, Qwen and Gemma. The most downloaded are huihui-ai/Qwen2.5-Coder-7B-Instruct-abliterated, huihui-ai/Qwen2.5-14B-Instruct-abliterated and TheDrummer/Rocinante-12B-v1.1.
The quants featherless-ai-quants publishes most often are Q8_0 (3092), Q6_K (3092), Q5_K_S (3092) and Q5_K_M (3092). Their Q4_K_M files range from 0.4 to 47.4 GB. Parameter counts run from 494M to 72.7B, and the longest trained context window is 1024K tokens.
Frequently asked questions
What GGUF models has featherless-ai-quants published?
3,092 models are listed here, mainly from the Llama, Qwen and Gemma families. The most downloaded is huihui-ai/Qwen2.5-Coder-7B-Instruct-abliterated.
Which quants does featherless-ai-quants publish?
The most common are Q8_0 (3092 models), Q6_K (3092 models), Q5_K_S (3092 models) and Q5_K_M (3092 models).