GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
LL
read1337 Llama 3.1 8B bnb 4bit marketing
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GR
Granite 7b Instruct
Granite · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
PatronusAI Llama 3 Patronus Lynx 70B Instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
MI
m8than mistral 7b instruct 0.2
Mistral · RichardErkhov
Params7.24B
Context—
Q4_K_M1.1 GB
ZE
Zephyr 7b Beta
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3 8B ArliAI Formax v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
NO
Noromaid-7B-0.4-DPO
Noromaid · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
NO
nothingiisreal MN 12B Starcannon v3
Nothingiisreal · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
LL
scb10x llama 3 typhoon v1.5x 70b instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
LL
LLaMA 3 8B SFR Iterative DPO R
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
VI
vicgalle ConfigurableHermes 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama 3 Instruct 8B SimPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
DE
Husky v1 Math Deepseekmath 7b Instruct
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
MI
nbeerbower bophades mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RA
RakutenAI 7B chat
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
CH
ChaoticNeutrals Prima LelantaclesV5 7b
Chaoticneutrals · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Nous Hermes 2 Mistral 7B DPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3 Swallow 70B v0.1
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
SP
SparseLlama 3 8B Pruned_50.2Of4
Sparsellama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
PO
Poro 34B
Poro · RichardErkhov
Params34B
Context7K
Q4_K_M22.4 GB
ME
MetaMath Bagel DPO 34B
Metamath · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
OL
OLMo 7B Instruct Hf
OLMo · RichardErkhov
Params6.89B
Context2K
Q4_K_M4.2 GB
VI
vinallama-2.7b
Vinallama · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.7 GB
YI
Yi 1.5 6B
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
VA
VAGOsolutions SauerkrautLM 7b HerO
Vagosolutions · RichardErkhov
Params3.8B
Context—
Q4_K_M2.2 GB
LL
microsoft LLaMA 2 7b GTL Delta
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
MI
Mixtral 8x7B Instruct v0.1
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
TY
Typhoon 7b Instruct 02 19 2024
Typhoon · RichardErkhov
Params7.27B
Context32K
Q4_K_M4.4 GB