GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
GE
Gemma 7b
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
YI
Yi-1.5-9B-32K
Yi · RichardErkhov
Params8.83B
Context32K
Q4_K_M5.3 GB
KU
kukulemon-spiked-9B
Kukulemon · RichardErkhov
Params8.99B
Context8K
Q4_K_M5.4 GB
QW
Qwen2.5 7B Instruct Uncensored
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
vicgalle Humanish Roleplay Llama 3.1 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama 2 7b Ebft Sparsegpt Outlier Wiki Block Outlier
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
vicgalle Configurable Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2 7B Instruct Step DPO
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
Meta Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
LL
llama2-7b-chat-hf-v3
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
SE
SeaLLMs v3 1.5B Chat
Seallms · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
LA
Lawma 8b
Lawma · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
TI
TinySlime-1.1B-v1.0
Tinyslime · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
nirosunid Llama 3.1 8B bnb 4bit python v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
ML
mlfoundations dev stackoverflow 10000tasks 0p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
jan hq Ichigo llama3.1 s instruct v0.4 step 7000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
ST
statking zephyr 7b sft full orpo
Statking · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
1 800 SHARED TASKS PHI3 Medium NLI 16bit
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.4 GB
DM
dmis lab meerkat 7b v1.0
Dmis · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 7B Instruct Abliterated v3
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
hf 100 Llama 3.1 Spellbound StoryWriter 70b instruct 0.3 16bit
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
BE
beomi Yi Ko 34B Chat Preview
Beomi · RichardErkhov
Params34.6B
Context2K
Q4_K_M20.8 GB
SA
saishf Top Western Maid 7B
Saishf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
DE
Deepseekmath 7b Sft MATH v2
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GO
Gonzo Code 7B
Gonzo · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
Granite 20b Code Instruct 8k
Granite · RichardErkhov
Params20.1B
Context8K
Q4_K_M12.8 GB
MI
Ft Mistral 7b Instruct v0.2 Sorry Bench 202406
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB