GGUF models by TheBloke
1062 GGUF models quantized by TheBloke on Hugging Face. Sizes run from 1.1B to 124B parameters. The longest trained context is 195K tokens.
MI
sciphi_sciphi-mistral-7b-32k
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Nous Hermes Llama2 70B
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
LL
harborwater_open-llama-3b-v2-wizard-evol-instuct-v2-196k
Llama · TheBloke
Params3.43B
Context2K
Q4_K_M2.6 GB
WI
ehartford_wizard-vicuna-7b-uncensored
Wizard · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
LL
ehartford_wizardlm-1.0-uncensored-codellama-34b
Llama · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
MI
migtissera_synthia-moe-v3-mixtral-8x7b
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
PY
Pygmalion 2 7B
Pygmalion · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
JA
japanese stablelm instruct beta 7B
Japanese · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
YI
ehartford_dolphin-2_2-yi-34b
Yi · TheBloke
Params34.4B
Context16K
Q4_K_M20.7 GB
OP
openchat_openchat-3.5-1210
Openchat · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
AI
airoboros l2 13b gpt4 2.0
Airoboros · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
OP
openbuddy_openbuddy-zephyr-7b-v14.1
Openbuddy · TheBloke
Params7.28B
Context32K
Q4_K_M4.4 GB
AI
jondurbin_airoboros-c34b-3.1.2
Airoboros · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
GE
Genz 70b
Genz · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
OR
ddh0_orcamaidxl-17b-32k
Orcamaidxl · TheBloke
Params17.5B
Context32K
Q4_K_M10.5 GB
MI
mlabonne_neuralhermes-2.5-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
WI
ehartford_wizardlm-7b-v1.0-uncensored
Wizardlm · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
DI
discoresearch_discolm-70b
Discolm · TheBloke
Params69B
Context8K
Q4_K_M41.4 GB
RW
rwitz_go-bruins-v2
Rwitz · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
AI
Airochronos L2 13B
Airochronos · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LL
ShiningValiantXS 1.1
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LL
phind_phind-codellama-34b-v1
Llama · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
LL
georgesung_llama2_7b_chat_uncensored
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
NE
nexusflow_nexusraven-v2-13b
Nexusraven · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
PH
phi 2 electrical engineering
Phi · TheBloke
Params2.78B
Context2K
Q4_K_M1.7 GB
MI
nondzu_mistral-7b-instruct-v0.2-code-ft
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
PY
Pygmalion 2 13B
Pygmalion · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LL
Llama 2 70B
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
MA
openaccess-ai-collective_manticore-13b-chat-pyg
Manticore · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
KA
KafkaLM 70B German V0.1
Kafkalm · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB