GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
read1337 Llama 3.1 8B bnb 4bit marketing
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GR
Granite 7b Instruct
Granite · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
PatronusAI Llama 3 Patronus Lynx 70B Instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
MI
m8than mistral 7b instruct 0.2
Mistral · RichardErkhov
Params7.24B
Context—
Q4_K_M1.1 GB
GGUF
ZE
Zephyr 7b Beta
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 8B ArliAI Formax v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
NO
Noromaid-7B-0.4-DPO
Noromaid · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NO
nothingiisreal MN 12B Starcannon v3
Nothingiisreal · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
scb10x llama 3 typhoon v1.5x 70b instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
LL
LLaMA 3 8B SFR Iterative DPO R
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
VI
vicgalle ConfigurableHermes 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama 3 Instruct 8B SimPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
DE
Husky v1 Math Deepseekmath 7b Instruct
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
MI
nbeerbower bophades mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RA
RakutenAI 7B chat
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
CH
ChaoticNeutrals Prima LelantaclesV5 7b
Chaoticneutrals · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Nous Hermes 2 Mistral 7B DPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Llama 3 Swallow 70B v0.1
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
SP
SparseLlama 3 8B Pruned_50.2Of4
Sparsellama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
PO
Poro 34B
Poro · RichardErkhov
Params34B
Context7K
Q4_K_M22.4 GB
GGUF
ME
MetaMath Bagel DPO 34B
Metamath · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
OL
OLMo 7B Instruct Hf
OLMo · RichardErkhov
Params6.89B
Context2K
Q4_K_M4.2 GB
GGUF
VI
vinallama-2.7b
Vinallama · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.7 GB
GGUF
YI
Yi 1.5 6B
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
VA
VAGOsolutions SauerkrautLM 7b HerO
Vagosolutions · RichardErkhov
Params3.8B
Context—
Q4_K_M2.2 GB
GGUF
LL
microsoft LLaMA 2 7b GTL Delta
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
MI
Mixtral 8x7B Instruct v0.1
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
TY
Typhoon 7b Instruct 02 19 2024
Typhoon · RichardErkhov
Params7.27B
Context32K
Q4_K_M4.4 GB
GGUF

Keep browsing