GGUF models by TheBloke
1062 GGUF models quantized by TheBloke on Hugging Face. Sizes run from 1.1B to 124B parameters. The longest trained context is 195K tokens.
About TheBloke on Hugging Face
TheBloke has published 1,062 GGUF models that ggufmodels.com tracks, across families such as Llama, Mistral and Airoboros. The most downloaded are tinyllama_tinyllama-1.1b-chat-v1.0, Llama 2 7B Chat and mistralai_mistral-7b-instruct-v0.2.
The quants TheBloke publishes most often are Q2_K (1058), Q3_K_M (1050), Q5_0 (1046) and Q4_0 (1046). Their Q4_K_M files range from 0.7 to 41.8 GB. Parameter counts run from 1.1B to 124B, and the longest trained context window is 195K tokens.
Frequently asked questions
What GGUF models has TheBloke published?
1,062 models are listed here, mainly from the Llama, Mistral and Airoboros families. The most downloaded is tinyllama_tinyllama-1.1b-chat-v1.0.
Which quants does TheBloke publish?
The most common are Q2_K (1058 models), Q3_K_M (1050 models), Q5_0 (1046 models) and Q4_0 (1046 models).