GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
CodeActAgent Llama 2 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
CodeLlama 13B Fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
BR
broalantap GPT2 large 4 16000steps
Broalantap · RichardErkhov
Params839M
Context1K
Q4_K_M0.5 GB
GGUF
TR
Tristan dclm fasttext oh eli5 1b
Tristan · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
MI
mlfoundations dev mistral alpaca sft sample 8
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
llama-3-8B-Instruct-function-calling-v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
GE
christinechagas gemma 2b instruct ft health bot
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
sanchit gandhi distil mistral 1.5B v0.1
Mistral · RichardErkhov
Params1.57B
Context32K
Q4_K_M1.0 GB
GGUF
QW
masa research Qwen2 0.5B 20240829 175401
Qwen · RichardErkhov
Params494M
Context128K
Q4_K_M0.4 GB
GGUF
XA
xammi final mp model merged
Xammi · RichardErkhov
Params1.1B
Context4K
Q4_K_M0.7 GB
GGUF
LL
llama-2-70B-instruct
Llama · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
MI
synapsellm-7b-mistral-v0.4-preview2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
tinyllama-1.1b-sum-dpo-full_LR1e-7_2epochs
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
MA
Marcoroni Neural Chat 7B v2
Marcoroni · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
juhwanlee llmdo Mistral 7B case 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev hp ablations mistral epoch3 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
20240105_mistral Step50
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
YI
hongzoh Yi 6B Open Platypus v2
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
PH
myfi parser phi 3.5 mini instruct epochs 4
Phi · RichardErkhov
Params3.82B
Context8K
Q4_K_M2.3 GB
GGUF
PY
Pythia 2.8b
Pythia · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
IN
inswave AISquare Instruct yi ko 6b v0.9.26
Inswave · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
BR
braindao iq code evmind 0.5b instruct v0.2411.3 193
Braindao · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
CodingTool use
MI
Agnuxo Mamba Codestral 7B Instruct CODE Python Spanish English Asistant 16bit v2
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
Coding
ZE
zephyr-7b-dpo-full
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Meta-Llama-3-8B-Instruct-dequantized
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ML
mlfoundations dev oh v1.3 evol instruct x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 Coder 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
CodingTool use
YU
yuntaeyang KoSOLAR 10.7B dpo v1.0
Yuntaeyang · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
GGUF

Keep browsing