GGUF models by TheBloke

1062 GGUF models quantized by TheBloke on Hugging Face. Sizes run from 1.1B to 124B parameters. The longest trained context is 195K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
sciphi_sciphi-mistral-7b-32k
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Nous Hermes Llama2 70B
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
Tool use
LL
harborwater_open-llama-3b-v2-wizard-evol-instuct-v2-196k
Llama · TheBloke
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
WI
ehartford_wizard-vicuna-7b-uncensored
Wizard · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
LL
ehartford_wizardlm-1.0-uncensored-codellama-34b
Llama · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
Coding
MI
migtissera_synthia-moe-v3-mixtral-8x7b
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
GGUF
PY
Pygmalion 2 7B
Pygmalion · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
JA
japanese stablelm instruct beta 7B
Japanese · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
YI
ehartford_dolphin-2_2-yi-34b
Yi · TheBloke
Params34.4B
Context16K
Q4_K_M20.7 GB
GGUF
OP
openchat_openchat-3.5-1210
Openchat · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
AI
airoboros l2 13b gpt4 2.0
Airoboros · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
OP
openbuddy_openbuddy-zephyr-7b-v14.1
Openbuddy · TheBloke
Params7.28B
Context32K
Q4_K_M4.4 GB
GGUF
AI
jondurbin_airoboros-c34b-3.1.2
Airoboros · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
GGUF
GE
Genz 70b
Genz · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
OR
ddh0_orcamaidxl-17b-32k
Orcamaidxl · TheBloke
Params17.5B
Context32K
Q4_K_M10.5 GB
GGUF
MI
mlabonne_neuralhermes-2.5-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
WI
ehartford_wizardlm-7b-v1.0-uncensored
Wizardlm · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
DI
discoresearch_discolm-70b
Discolm · TheBloke
Params69B
Context8K
Q4_K_M41.4 GB
Tool use
RW
rwitz_go-bruins-v2
Rwitz · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
AI
Airochronos L2 13B
Airochronos · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
ShiningValiantXS 1.1
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
phind_phind-codellama-34b-v1
Llama · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
Coding
LL
georgesung_llama2_7b_chat_uncensored
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
NE
nexusflow_nexusraven-v2-13b
Nexusraven · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
GGUF
PH
phi 2 electrical engineering
Phi · TheBloke
Params2.78B
Context2K
Q4_K_M1.7 GB
GGUF
MI
nondzu_mistral-7b-instruct-v0.2-code-ft
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
PY
Pygmalion 2 13B
Pygmalion · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
Llama 2 70B
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
MA
openaccess-ai-collective_manticore-13b-chat-pyg
Manticore · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
Coding
KA
KafkaLM 70B German V0.1
Kafkalm · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF

Keep browsing