GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Hermes 2 Theta Llama 3 70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
Tool use
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
OA
h2oai h2o danube2 1.8b sft
Oai · RichardErkhov
Params1.83B
Context8K
Q4_K_M1.1 GB
GGUF
IA
iamspathan half girlfriend gpt2
Iamspathan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
s3nh nsfw noromaid mistral instruct
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SH
Shieldgemma 2b
Shieldgemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
DP
Yi Ko 6B DPO v2
Dpo · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
MI
Mixtral_7Bx2_MoE_13B
Mistral · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
NO
nothingiisreal L3 8B Celeste v1
Nothingiisreal · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
AL
allknowingroger MultiverseEx26 Neurallaymons 12B MoE
Allknowingroger · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
GE
yhkim9362 gemma en ko 7b v0.1
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
DE
yiran wang3 clean mining sft split deepseek packing sft
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
AS
asiansoul SmartLlama 3 Ko 8B 256k PoSE
Asiansoul · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
thusinh1969 Guardian V0.1 LLaMA3.1 8B 5G 6Oct2024 epoch1.3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
GE
AXCXEPT EZO Humanities 9B gemma 2 it
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
SA
saucam Arithmo Wizard 2 7B
Saucam · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Meta-Llama-3-70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IN
InternLM2
InternLM · RichardErkhov
Params1.89B
Context32K
Q4_K_M1.2 GB
GGUF
MI
Mistral 7B v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
VI
Vidyayini gita text generation gpt2
Vidyayini · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
openinstruct-mistral-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
OpenELM 1_1B DPO Full 2 5
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
ST
Starcoder2 3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
DA
H2O Danube3 500m Base
Danube · RichardErkhov
Params514M
Context8K
Q4_K_M0.3 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
ricepaper vi gemma 2b RAG
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
TI
Tiny-Cowboy-1.1b-v0.1
Tiny · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
GE
Lion Gemma 7b Cn
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF

Keep browsing