GGUF models by LiteLLMs
73 GGUF models quantized by LiteLLMs on Hugging Face. Sizes run from 272M to 141B parameters. The longest trained context is 1024K tokens.
About LiteLLMs on Hugging Face
LiteLLMs has published 73 GGUF models that ggufmodels.com tracks, across families such as Llama, Mistral and Openelm. The most downloaded are Mixtral 8x22B v0.1, Llama 3 Open Ko 8B and Meta Llama 3 70B Instruct.
The quants LiteLLMs publishes most often are Q5_K_S (73), Q5_K_M (73), Q5_0 (73) and Q4_K_S (73). Their Q4_K_M files range from 0.2 to 85.6 GB. Parameter counts run from 272M to 141B, and the longest trained context window is 1024K tokens.
Frequently asked questions
What GGUF models has LiteLLMs published?
73 models are listed here, mainly from the Llama, Mistral and Openelm families. The most downloaded is Mixtral 8x22B v0.1.
Which quants does LiteLLMs publish?
The most common are Q5_K_S (73 models), Q5_K_M (73 models), Q5_0 (73 models) and Q4_K_S (73 models).