GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
llama-3-typhoon-v1.5-8b
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
flammenai flammen9 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
XL
xLAM 8x7b R
Xlam · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
Tool use
KR
kromeurus L3.1 Ablaze Vulca v0.1 8B
Kromeurus · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
NE
Nekomata 7b Pfn Qfin
Nekomata · RichardErkhov
Params7.72B
Context32K
Q4_K_M4.9 GB
GGUF
QW
Firefly Qwen1.5 En 7b
Qwen · RichardErkhov
Params7.72B
Context32K
Q4_K_M4.8 GB
GGUF
TU
tulu-2-dpo-70b
Tulu · RichardErkhov
Params69B
Context8K
Q4_K_M41.4 GB
GGUF
NO
notHumpback-M0
Nothumpback · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
LL
Llama 3 Swallow 8B v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SE
SeaLLMs v3 1.5B
Seallms · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
UM
v000000 L3 Umbral Storm 8B t0.0001
Umbral · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ST
StructLM 7B
Structlm · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
LL
Llama 3 Instruct 8B SimPO v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
CH
Chatntq Ja 7b v1.0
Chatntq · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
LL
AIML GEEK prompt finetuned CodeLlama7b Instruct
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
cais HarmBench Llama 2 13b cls
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
JS
jstredacted gamma 2b instruct ft awesome chatgpt prompts
Jstredacted · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
YU
Yuma42 KangalKhan RawRuby 7B
Yuma · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
EM
emailgen-pythia-410m-deduped
Emailgen · RichardErkhov
Params405M
Context2K
Q4_K_M0.3 GB
GGUF
PH
TroyDoesAI Phi 3 Context Obedient RAG
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama Phishsense 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
NO
nothingiisreal L3 8B Instruct Abliterated DWP
Nothingiisreal · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 72B
Qwen · RichardErkhov
Params72.7B
Context128K
Q4_K_M47.4 GB
Tool use
LL
laislemke LLaMA 2 vicuna 7b slerp
Llama · RichardErkhov
Params6.74B
Context32K
Q4_K_M0.6 GB
GGUF
LL
Evolved Llama3 ELYZA Uncensored 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PY
pygmalion-1.3b
Pygmalion · RichardErkhov
Params1.41B
Context2K
Q4_K_M0.9 GB
GGUF
YU
yuntian deng gpt2 explicit cot multiplication 20 digits
Yuntian · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
LL
DataPilot Llama3.1 ArrowSE v0.4
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF

Keep browsing