GGUF models by TheBloke

1062 GGUF models quantized by TheBloke on Hugging Face. Sizes run from 1.1B to 124B parameters. The longest trained context is 195K tokens.

Most downloadedNewestLargestLongest contextSmallest
MY
L2 MythoMax22b Instruct Falseblock
Mythomax · TheBloke
Params21.8B
Context4K
Q4_K_M13.2 GB
GGUF
MI
brillibits_instruct_mixtral-8x7b-v0.1_dolly15k
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
GGUF
MI
azure99_blossom-v3-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
YI
mlinmg_sg-raccoon-yi-55b-200k
Yi · TheBloke
Params55.6B
Context195K
Q4_K_M33.3 GB
GGUF
TU
allenai_tulu-2-7b
Tulu · TheBloke
Params6.74B
Context8K
Q4_K_M4.1 GB
GGUF
QC
qCammel 13
Qcammel · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
vietnamese llama2 7B 40GB
Llama · TheBloke
Params6.86B
Context4K
Q4_K_M4.2 GB
GGUF
WE
senseable_westlake-7b-v2
Westlake · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
WI
wizardlm_wizardmath-7b-v1.1
Wizardmath · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
flozi00_llama-2-13b-german-assistant-v2
Llama · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
MI
orangetin_openhermes-mixtral-8x7b
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
Tool use
LL
ehartford_dolphin-llama-13b
Llama · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
GO
gorilla openfunctions v1
Gorilla · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
BA
weyaxi_bagel-hermes-2x34b
Bagel · TheBloke
Params60.8B
Context195K
Q4_K_M36.7 GB
Tool use
DE
hkust-nlp_deita-7b-v1.0-sft
Deita · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LO
sanjiwatsuki_loyal-macaroni-maid-7b
Loyal · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
fpham_karen_theeditor_v2_creative_mistral_7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PA
janhq_pandora-v1-10.7b
Pandora · TheBloke
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
MI
qblocks_mistral_7b_norobots
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
YI
blossom v3 1 yi 34b
Yi · TheBloke
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
XW
Xwin LM 7B V0.1
Xwin · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
ShiningValiantXS
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
SA
Samantha 1.11 70B
Samantha · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
DO
dolphin 2.2 70B
Dolphin · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
MI
nickynicky_mistral-7b-openorca-oasst_top1_2023-08-25-v1
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OR
OrcaMaid 13B
Orcamaid · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
FR
fredithefish_madmix-v0.2
Fredithefish · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
KI
Rose Kimiko 20B
Kimi · TheBloke
Params20B
Context4K
Q4_K_M12.0 GB
GGUF
UN
fblgit_una-thebeagle-7b-v1
Una · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NA
Nanbeige 16B Base
Nanbeige · TheBloke
Params15.8B
Context4K
Q4_K_M9.6 GB
Coding

Keep browsing