GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
TI
TinyLlama-1.1B-intermediate-step-240k-503b
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
RU
RuishiCh0314 code mix model finetuned
Ruishich · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
ST
starcoder2-7b
StarCoder · RichardErkhov
Params7.17B
Context16K
Q4_K_M4.5 GB
KU
Kunoichi-DPO-7B
Kunoichi · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
ML
mlfoundations dev oh teknium scaling down ratiocontrolled 0.6
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
mlfoundations dev llama3 1 8b oh v3.1 wo collective cognition
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
ST
starcoder2-15b-instruct-v0.1
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
ML
mlfoundations dev oh v1.3 camel chemistry x.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Bllossom70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
SM
SmolLlama-1.5B
Smolllama · RichardErkhov
Params1.54B
Context2K
Q4_K_M0.9 GB
MI
Mistral 7B Instruct V0.2_Bio Tutor_Dpo
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
smol_llama-220M-GQA-fineweb_edu
Llama · RichardErkhov
Params218M
Context2K
Q4_K_M0.1 GB
MY
Myrrh_solar_10.7b_3.0
Myrrh · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
NO
NorLlama 3B
Norllama · RichardErkhov
Params3.07B
Context2K
Q4_K_M3.0 GB
LL
formapproval llama 8b WEbLINX ft WebLINX v2 vllm
Llama · RichardErkhov
Params2.7B
Context4K
Q4_K_M1.6 GB
LL
mlfoundations dev hp ablations llama3 epoch2 dcftv1.2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Hermes 2 Pro Llama 3 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
HE
Heejindo rationale model e3 save5000 f2
Heejindo · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
ML
mlfoundations dev OH original wo caseus custom
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
RA
RakutenAI 7B
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
DE
deepseek-coder-6.7b-chat
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
QW
natong19 Qwen2 7B Instruct abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
JA
Jais Adapted 7b
Jais · RichardErkhov
Params7B
Context4K
Q4_K_M4.3 GB
LL
Llama 2 7b Hf Llama2 Chat 80k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
pantelis ninja unsloth Qwen2.5 3B Instruct dtype bfloat16 r 64 lr 0.0007
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
LL
Meta Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
CA
Cartinoe5930 KoRAE 13b
Cartinoe · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
LL
LDCC Instruct Llama 2 Ko 13B v4
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
GE
Google_Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
PH
IntelLabs sqft phi 3 mini 4k 50 base
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB