GGUF models by TheBloke

1062 GGUF models quantized by TheBloke on Hugging Face. Sizes run from 1.1B to 124B parameters. The longest trained context is 195K tokens.

Most downloadedNewestLargestLongest contextSmallest
TE
migtissera_tess-m-v1.1
Tess · TheBloke
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
MI
embeddedllm_mistral-7b-merge-14-v0.1
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Embedding
LL
Llama2 Chat AYT 13B
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
HY
digitous_13b-hypermantis
Hypermantis · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
Roleplay
MI
ehartford_samantha-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SW
Swallow 13B Instruct
Swallow · TheBloke
Params13.1B
Context4K
Q4_K_M7.9 GB
GGUF
LL
LLaMA2 13B Estopia
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LE
leolm_leo-hessianai-70b
Leo · TheBloke
Params69B
Context8K
Q4_K_M41.4 GB
GGUF
LU
Lumosia MoE 4x10.7
Lumosia · TheBloke
Params36B
Context4K
Q4_K_M20.4 GB
Roleplay
QU
quantumaikr_quantum-v0.01
Quantum · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MY
gryphe_mythomist-7b
Mythomist · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
PlatYi 34B Llama Q v3
Llama · TheBloke
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
SA
Samantha 1.11 13B
Samantha · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
KI
Mythalion Kimiko v2
Kimi · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
sheep duck llama 2 70B v1.1
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
LL
rombodawg_losslessmegacoder-llama2-13b-mini
Llama · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
Coding
AI
airoboros l2 13b gpt4 m2.0
Airoboros · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
KA
keynote-technology_kai-7b-instruct
Kai · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
LL
nousresearch_yarn-llama-2-7b-128k
Llama · TheBloke
Params6.74B
Context128K
Q4_K_M4.1 GB
GGUF
SP
Spicyboros 13B 2.2
Spicyboros · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
ST
carperai_stable-vicuna-13b-delta
Stable · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
LL
toxicqa Llama2 13B
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
GR
v1olet_v1olet_merged_dpo_7b_v3
Greennode · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PI
maywell_pivot-0.1-evil-a
Pivot · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ZA
Zarablend MX L2 7B
Zarablend · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
openbuddy_openbuddy-mistral-7b-v13
Mistral · TheBloke
Params7.28B
Context32K
Q4_K_M4.4 GB
GGUF
UN
fblgit_unaversal-8x7b-v1beta
Unaversal · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
GGUF
WI
wizardlm_wizardmath-7b-v1.0
Wizardlm · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
XW
xwin-lm_xwincoder-13b
Xwincoder · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
Coding
LZ
lzlv 70B
Lzlv · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF

Keep browsing