GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
ZE
zephyr-7b-dpo-full-prompt-extend-chosen-delete_0.1
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
simonycl llama 3 8b instruct armorm judge
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
CR
crestf411 daybreak kunoichi 2dpo 7b
Crestf · RichardErkhov
Params7.24B
Context16K
Q4_K_M4.4 GB
GGUF
LL
Llama-3-Base-8B-SFT
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
dolphin-2.9.3-qwen2-1.5b
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
LL
LLM LAT llama2 7b chat lat removed backdoor5
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
SA
SambaLingo Slovenian Chat
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
GGUF
AI
AIFT Ko Orca Plat Yi Ko 6b v1.2 Dpo
Aift · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
LL
shawon majid llama 3 8B Instruct test smell detection raw
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
AP
Apollo 1.8B
Apollo · RichardErkhov
Params1.84B
Context8K
Q4_K_M1.2 GB
GGUF
JA
Jais Adapted 70b
Jais · RichardErkhov
Params69.5B
Context4K
Q4_K_M41.8 GB
GGUF
IN
InternLM2
InternLM · RichardErkhov
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh v3.1 wo gpt4 llm
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
BL
Blexus awareness test
Blexus · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
GR
Grafted Hermetic Platypus D 2x7B
Grafted · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
ML
mlfoundations dev oh teknium scaling down random 0.3
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Mistral 7B Merge 14 v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Embedding
TW
Twice-KoSOLAR-16.1B-instruct-test
Twice · RichardErkhov
Params16.1B
Context4K
Q4_K_M9.7 GB
GGUF
ST
Stopwolf DistilabelCerberus 7B slerp
Stopwolf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama3 Aloe 8B Alpha
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
IN
interview eval zephyr 7b math case 4
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
flozi00 Llama 2 13b german assistant v5
Llama · RichardErkhov
Params13.1B
Context2K
Q4_K_M7.9 GB
GGUF
ML
mlfoundations dev oh v1 2 only alpaca
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
gradientai Llama 3 70B Instruct Gradient 262k
Llama · RichardErkhov
Params70.6B
Context256K
Q4_K_M42.5 GB
GGUF
OP
OpenLLM Ro RoLlama3 8b Instruct
Openllm · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
inswave AISquare Instruct llama2 koen 13b v0.9.21
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
WE
Weyaxi OrcaMini Platypus2 13B QLoRA 0.80 epoch
Weyaxi · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
AH
Ahma 3B
Ahma · RichardErkhov
Params3.63B
Context2K
Q4_K_M2.8 GB
GGUF

Keep browsing