GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
GE
Gemma 7b
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
YI
Yi-1.5-9B-32K
Yi · RichardErkhov
Params8.83B
Context32K
Q4_K_M5.3 GB
GGUF
KU
kukulemon-spiked-9B
Kukulemon · RichardErkhov
Params8.99B
Context8K
Q4_K_M5.4 GB
GGUF
QW
Qwen2.5 7B Instruct Uncensored
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
vicgalle Humanish Roleplay Llama 3.1 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Roleplay
LL
Llama 2 7b Ebft Sparsegpt Outlier Wiki Block Outlier
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
vicgalle Configurable Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2 7B Instruct Step DPO
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Meta Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
LL
llama2-7b-chat-hf-v3
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
SE
SeaLLMs v3 1.5B Chat
Seallms · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
LA
Lawma 8b
Lawma · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
TI
TinySlime-1.1B-v1.0
Tinyslime · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
nirosunid Llama 3.1 8B bnb 4bit python v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
ML
mlfoundations dev stackoverflow 10000tasks 0p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
jan hq Ichigo llama3.1 s instruct v0.4 step 7000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
ST
statking zephyr 7b sft full orpo
Statking · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
1 800 SHARED TASKS PHI3 Medium NLI 16bit
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.4 GB
GGUF
DM
dmis lab meerkat 7b v1.0
Dmis · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B Instruct Abliterated v3
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
LL
hf 100 Llama 3.1 Spellbound StoryWriter 70b instruct 0.3 16bit
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
BE
beomi Yi Ko 34B Chat Preview
Beomi · RichardErkhov
Params34.6B
Context2K
Q4_K_M20.8 GB
GGUF
SA
saishf Top Western Maid 7B
Saishf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
DE
Deepseekmath 7b Sft MATH v2
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
GO
Gonzo Code 7B
Gonzo · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
GR
Granite 20b Code Instruct 8k
Granite · RichardErkhov
Params20.1B
Context8K
Q4_K_M12.8 GB
Coding
MI
Ft Mistral 7b Instruct v0.2 Sorry Bench 202406
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Keep browsing