GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
LL
CodeActAgent Llama 2 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
CodeLlama 13B Fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
BR
broalantap GPT2 large 4 16000steps
Broalantap · RichardErkhov
Params839M
Context1K
Q4_K_M0.5 GB
TR
Tristan dclm fasttext oh eli5 1b
Tristan · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
MI
mlfoundations dev mistral alpaca sft sample 8
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
llama-3-8B-Instruct-function-calling-v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
christinechagas gemma 2b instruct ft health bot
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
MI
sanchit gandhi distil mistral 1.5B v0.1
Mistral · RichardErkhov
Params1.57B
Context32K
Q4_K_M1.0 GB
QW
masa research Qwen2 0.5B 20240829 175401
Qwen · RichardErkhov
Params494M
Context128K
Q4_K_M0.4 GB
XA
xammi final mp model merged
Xammi · RichardErkhov
Params1.1B
Context4K
Q4_K_M0.7 GB
LL
llama-2-70B-instruct
Llama · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
MI
synapsellm-7b-mistral-v0.4-preview2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TI
tinyllama-1.1b-sum-dpo-full_LR1e-7_2epochs
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
MA
Marcoroni Neural Chat 7B v2
Marcoroni · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
juhwanlee llmdo Mistral 7B case 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mlfoundations dev hp ablations mistral epoch3 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
20240105_mistral Step50
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
YI
hongzoh Yi 6B Open Platypus v2
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
PH
myfi parser phi 3.5 mini instruct epochs 4
Phi · RichardErkhov
Params3.82B
Context8K
Q4_K_M2.3 GB
PY
Pythia 2.8b
Pythia · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
IN
inswave AISquare Instruct yi ko 6b v0.9.26
Inswave · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
BR
braindao iq code evmind 0.5b instruct v0.2411.3 193
Braindao · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
MI
Agnuxo Mamba Codestral 7B Instruct CODE Python Spanish English Asistant 16bit v2
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
ZE
zephyr-7b-dpo-full
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Meta-Llama-3-8B-Instruct-dequantized
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
ML
mlfoundations dev oh v1.3 evol instruct x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2.5 Coder 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
YU
yuntaeyang KoSOLAR 10.7B dpo v1.0
Yuntaeyang · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB