GGUF models by mlabonne
38 GGUF models quantized by mlabonne on Hugging Face. Sizes run from 596M to 70.6B parameters. The longest trained context is 128K tokens.
About mlabonne on Hugging Face
mlabonne has published 38 GGUF models that ggufmodels.com tracks, across families such as Gemma, Llama and Mistral. The most downloaded are Meta Llama 3.1 8B Instruct Abliterated, Gemma 3 27b It Abliterated and Gemma 3 12b It Abliterated v2.
The quants mlabonne publishes most often are Q8_0 (34), Q5_K_M (31), Q4_K_M (30) and Q6_K (25). Their Q4_K_M files range from 0.4 to 42.5 GB. Parameter counts run from 596M to 70.6B, and the longest trained context window is 128K tokens.
Frequently asked questions
What GGUF models has mlabonne published?
38 models are listed here, mainly from the Gemma, Llama and Mistral families. The most downloaded is Meta Llama 3.1 8B Instruct Abliterated.
Which quants does mlabonne publish?
The most common are Q8_0 (34 models), Q5_K_M (31 models), Q4_K_M (30 models) and Q6_K (25 models).