GGUF models by TheBloke
1062 GGUF models quantized by TheBloke on Hugging Face. Sizes run from 1.1B to 124B parameters. The longest trained context is 195K tokens.
WH
whiterabbitneo_whiterabbitneo-13b
Whiterabbitneo · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
AI
Airoboros L2 7B 2.1
Airoboros · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
codellama_codellama-7b-hf
Llama · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
MI
cognitivecomputations_dolphin-2.6-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 2 70B Chat
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
AT
Athena v1
Athena · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
AI
Airoboros L2 13B 2.1
Airoboros · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LL
Llama 2 13B chat
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
MI
nousresearch_nous-hermes-2-mixtral-8x7b-dpo
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M28.4 GB
DE
deepseek-ai_deepseek-coder-33b-base
DeepSeek · TheBloke
Params33.3B
Context16K
Q4_K_M19.9 GB
PU
PuddleJumper 13B
Puddlejumper · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
ST
berkeley-nest_starling-lm-7b-alpha
Starling · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
codellama_codellama-34b-hf
Llama · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
MI
teknium_openhermes-2-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
KI
MythoMax L2 Kimiko v2 13B
Kimi · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
MI
ehartford_dolphin-2.2.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
MY
Mythical Destroyer V2 L2 13B
Mythical · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
YI
brucethemoose_yi-34b-200k-dare-megamerge-v8
Yi · TheBloke
Params34.4B
Context195K
Q4_K_M20.7 GB
ZE
huggingfaceh4_zephyr-7b-beta
Zephyr · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
deepseek llm 7B chat
DeepSeek · TheBloke
Params6.91B
Context4K
Q4_K_M4.2 GB
DE
deepseek-ai_deepseek-coder-5.7bmqa-base
DeepSeek · TheBloke
Params5.7B
Context16K
Q4_K_M3.4 GB
DE
deepseek-ai_deepseek-coder-1.3b-base
DeepSeek · TheBloke
Params1.35B
Context16K
Q4_K_M0.9 GB
WI
wizardlm_wizardcoder-python-13b-v1.0
Wizardcoder · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
LL
Speechless Llama2 Hermes Orca Platypus WizardLM 13B
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LL
tap-m_luna-ai-llama2-uncensored
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
LL
codellama_codellama-34b-python-hf
Llama · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
MY
Mythical Destroyer L2 13B
Mythical · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
MI
ehartford_dolphin-2.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
AI
Airoboros L2 7B 2.2
Airoboros · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
Llama2 70B OASST SFT v10
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB