GGUF models by TheBloke

1062 GGUF models quantized by TheBloke on Hugging Face. Sizes run from 1.1B to 124B parameters. The longest trained context is 195K tokens.

Most downloadedNewestLargestLongest contextSmallest
TI
tinyllama_tinyllama-1.1b-chat-v1.0
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
LL
Llama 2 7B Chat
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
mistralai_mistral-7b-instruct-v0.2
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
deepseek-ai_deepseek-coder-6.7b-instruct
DeepSeek · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
mistralai_mistral-7b-instruct-v0.1
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
py007_tinyllama-1.1b-chat-v0.3
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
MY
MythoMax L2 13B
Mythomax · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
mistralai_mixtral-8x7b-instruct-v0.1
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
GGUF
LL
codellama_codellama-7b-instruct-hf
Llama · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Llama 2 7B
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
WI
ehartford_wizardlm-7b-uncensored
Wizardlm · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
DE
deepseek-ai_deepseek-coder-33b-instruct
DeepSeek · TheBloke
Params33.3B
Context16K
Q4_K_M19.9 GB
Coding
MI
open-orca_mistral-7b-openorca
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
deepseek-ai_deepseek-coder-1.3b-instruct
DeepSeek · TheBloke
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
MI
ehartford_dolphin-2.5-mixtral-8x7b
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
CodingTool use
PH
phi 2
Phi · TheBloke
Params2.78B
Context2K
Q4_K_M1.8 GB
Coding
LL
openassistant_codellama-13b-oasst-sft-v10
Llama · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
Coding
LL
codellama_codellama-34b-instruct-hf
Llama · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
Coding
MI
mistralai_mistral-7b-v0.1
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
codellama_codellama-13b-instruct-hf
Llama · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
Coding
CA
causallm_7b
TheBloke
Params7.72B
Context8K
Q4_K_M4.8 GB
Tool useVision
WI
wizardlm_wizardcoder-python-34b-v1.0
Wizardcoder · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
Coding
DE
deepseek-ai_deepseek-coder-6.7b-base
DeepSeek · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
cognitivecomputations_dolphin-2.7-mixtral-8x7b
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
CodingTool use
LL
jarradh_llama2_70b_chat_uncensored
Llama · TheBloke
Params69B
Context2K
Q4_K_M41.4 GB
GGUF
MI
teknium_openhermes-2.5-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MI
mistralai_mixtral-8x7b-v0.1
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
GGUF
AI
Airoboros L2 70b 2.2
Airoboros · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
WI
ehartford_wizard-vicuna-13b-uncensored
Wizard · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
LL
WizardLM 1.0 Uncensored Llama2 13B
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF

About TheBloke on Hugging Face

TheBloke has published 1,062 GGUF models that ggufmodels.com tracks, across families such as Llama, Mistral and Airoboros. The most downloaded are tinyllama_tinyllama-1.1b-chat-v1.0, Llama 2 7B Chat and mistralai_mistral-7b-instruct-v0.2.

The quants TheBloke publishes most often are Q2_K (1058), Q3_K_M (1050), Q5_0 (1046) and Q4_0 (1046). Their Q4_K_M files range from 0.7 to 41.8 GB. Parameter counts run from 1.1B to 124B, and the longest trained context window is 195K tokens.

Frequently asked questions

What GGUF models has TheBloke published?

1,062 models are listed here, mainly from the Llama, Mistral and Airoboros families. The most downloaded is tinyllama_tinyllama-1.1b-chat-v1.0.

Which quants does TheBloke publish?

The most common are Q2_K (1058 models), Q3_K_M (1050 models), Q5_0 (1046 models) and Q4_0 (1046 models).

Keep browsing