GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
CO
ConvAI-9b
Convai · RichardErkhov
Params8.99B
Context32K
Q4_K_M5.4 GB
PA
pankajmathur orca mini 3b
Pankajmathur · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
WI
WizardVicuna Uncensored 3B 0719
Wizardvicuna · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
BH
bhenrym14 airoboros 33b gpt4 1.4.1 PI 8192 fp16
Bhenrym · RichardErkhov
Params32.5B
Context2K
Q4_K_M19.6 GB
JW
jwkweon CUBOX SOLAR 10.7B DPO v0.1
Jwkweon · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.5 GB
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr1e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 3B Instruct JaMagpie Preview
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
LL
gpjt Qwen1.5 0.5B openassistant guanaco llama2 format
Llama · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
GE
Gemma 2b It Bnb 4bit
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
MI
disi unibo nlp mistral SFT medqa triples nographeval cot
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
BH
bhavinjawade SOLAR 10B OrcaDPO Jawade
Bhavinjawade · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
ML
mlfoundations dev oh v1.3 slim orca x.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DA
DavidAU MN Dark Planet TITAN 12B
Davidau · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
ML
mlfoundations dev oh v1.3 evol instruct x8
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
dolphin-2.9.2-qwen2-7b
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
BL
v000000 L3 8B BlueSerpentine
Blueserpentine · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
UN
uniwiz-7B-v0.1
Uniwiz · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Agnuxo Llama 3.1 Minitron 4B Instruct CODE Python Spanish English Asistant 16bit v2
Llama · RichardErkhov
Params4.54B
Context128K
Q4_K_M2.8 GB
LL
MMed Llama 3 8B EnIns
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
TO
tongliuphysics zephyr 7b ultra p 0.07
Tongliuphysics · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TU
Tulu v2.5 Dpo 13b Uf Mean
Tulu · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
FR
Fredithefish Guanaco 3B Uncensored
Fredithefish · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
SU
suehyunpark potpourri 8b inst fft induction bc trajectory max10 per task warm start
Suehyunpark · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
TI
TinyLlama-1.1B-Chat-v0.4
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
MI
osanseviero mistral instruct frankenmerge
Mistral · RichardErkhov
Params8.99B
Context32K
Q4_K_M5.4 GB
LZ
lzw1008 Emollama chat 7b
Lzw · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
PY
Pythia 1.4b
Pythia · RichardErkhov
Params1.41B
Context2K
Q4_K_M0.9 GB
RE
reflect_llm8B_om2-mstlrg300k460kLlm33130k_SftDpo-MgSpsdpIter2T1_b0.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB