GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
LL
ibivibiv Llama 3 2B Instruct QuarterWit
Llama · RichardErkhov
Params2.8B
Context8K
Q4_K_M1.8 GB
LL
uukuguy speechless codellama orca airoboros 13b 0.10e
Llama · RichardErkhov
Params410M
Context—
Q4_K_M2.5 GB
MI
ewqr2130 mistral inst v02 dpo
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
mlfoundations dev mistral 7b 0 3 oh dcft v3.1 llama 3.3 70b
Llama · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
QW
VanillaKD Pretrain Qwen 500M
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
LL
codellama CodeLlama 13b hf
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
OC
OceanGPT 2B v0.1
Oceangpt · RichardErkhov
Params2.72B
Context4K
Q4_K_M1.8 GB
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GE
bitshrine gemma 2 2B it hermes function calling
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GP
ST23423 gpt2 pretrained custom eli5tech
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
QW
Qwen2 Math 7b Sft OpenHermes2.5_continue_with_WebInstructSub Checkpoint 1000
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
UN
Unbabel TowerBase 7B v0.1
Unbabel · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
cognitivecomputations dolphin 2.1 mistral 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
mlfoundations dev llama3 1 8b oh dcft v1 wo evol instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
CodeLlama-7B-fp16
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
TR
Trendyol LLM 7b Chat v1.8
Trendyol · RichardErkhov
Params7.34B
Context32K
Q4_K_M4.4 GB
XU
XuanYuan 13B Chat
Xuanyuan · RichardErkhov
Params13.1B
Context8K
Q4_K_M7.9 GB
MI
clowman openchat mistral 7b reproduce
Mistral · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
MI
pszemraj Mistral v0.3 6B
Mistral · RichardErkhov
Params5.94B
Context32K
Q4_K_M3.6 GB
TI
TinyLlama-1.1B-Chat-v1.0-solidworks-intro
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
YI
Faro Yi 34B DPO
Yi · RichardErkhov
Params34.4B
Context32K
Q4_K_M20.7 GB
LL
Llama-3.2-3B-Instruct-cot-19-11-5
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
AL
allknowingroger MultiMash 12B slerp
Allknowingroger · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
ME
mergekit community final test 4 v2 original old recipe humanish base
Mergekit · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
mlfoundations dev llama3 1 8b oh v3.1 wo glaive code assistant
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
llama-traditional-chinese-120M
Llama · RichardErkhov
Params120M
Context2K
Q4_K_M0.1 GB
LL
BEGADE llama 3.2 3b it ChatBot
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
RI
101M 0.4
Richarderkhov · RichardErkhov
Params137M
Context1K
Q4_K_M0.1 GB
MI
vicgalle Configurable Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
cais HarmBench Mistral 7b val cls
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB