GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
TI
TinyLlama-1.1B-intermediate-step-240k-503b
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
RU
RuishiCh0314 code mix model finetuned
Ruishich · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
Coding
ST
starcoder2-7b
StarCoder · RichardErkhov
Params7.17B
Context16K
Q4_K_M4.5 GB
Coding
KU
Kunoichi-DPO-7B
Kunoichi · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
ML
mlfoundations dev oh teknium scaling down ratiocontrolled 0.6
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh v3.1 wo collective cognition
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
ST
starcoder2-15b-instruct-v0.1
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
Coding
ML
mlfoundations dev oh v1.3 camel chemistry x.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Bllossom70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
SM
SmolLlama-1.5B
Smolllama · RichardErkhov
Params1.54B
Context2K
Q4_K_M0.9 GB
GGUF
MI
Mistral 7B Instruct V0.2_Bio Tutor_Dpo
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
smol_llama-220M-GQA-fineweb_edu
Llama · RichardErkhov
Params218M
Context2K
Q4_K_M0.1 GB
GGUF
MY
Myrrh_solar_10.7b_3.0
Myrrh · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
NO
NorLlama 3B
Norllama · RichardErkhov
Params3.07B
Context2K
Q4_K_M3.0 GB
GGUF
LL
formapproval llama 8b WEbLINX ft WebLINX v2 vllm
Llama · RichardErkhov
Params2.7B
Context4K
Q4_K_M1.6 GB
GGUF
LL
mlfoundations dev hp ablations llama3 epoch2 dcftv1.2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Hermes 2 Pro Llama 3 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
HE
Heejindo rationale model e3 save5000 f2
Heejindo · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
ML
mlfoundations dev OH original wo caseus custom
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
RA
RakutenAI 7B
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
DE
deepseek-coder-6.7b-chat
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
QW
natong19 Qwen2 7B Instruct abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
JA
Jais Adapted 7b
Jais · RichardErkhov
Params7B
Context4K
Q4_K_M4.3 GB
GGUF
LL
Llama 2 7b Hf Llama2 Chat 80k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
pantelis ninja unsloth Qwen2.5 3B Instruct dtype bfloat16 r 64 lr 0.0007
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
GGUF
LL
Meta Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
CA
Cartinoe5930 KoRAE 13b
Cartinoe · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
LL
LDCC Instruct Llama 2 Ko 13B v4
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
GE
Google_Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
PH
IntelLabs sqft phi 3 mini 4k 50 base
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF

Keep browsing