GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
CO
ConvAI-9b
Convai · RichardErkhov
Params8.99B
Context32K
Q4_K_M5.4 GB
GGUF
PA
pankajmathur orca mini 3b
Pankajmathur · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
WI
WizardVicuna Uncensored 3B 0719
Wizardvicuna · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
BH
bhenrym14 airoboros 33b gpt4 1.4.1 PI 8192 fp16
Bhenrym · RichardErkhov
Params32.5B
Context2K
Q4_K_M19.6 GB
GGUF
JW
jwkweon CUBOX SOLAR 10.7B DPO v0.1
Jwkweon · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.5 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr1e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 3B Instruct JaMagpie Preview
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
LL
gpjt Qwen1.5 0.5B openassistant guanaco llama2 format
Llama · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
GGUF
GE
Gemma 2b It Bnb 4bit
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
disi unibo nlp mistral SFT medqa triples nographeval cot
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
BH
bhavinjawade SOLAR 10B OrcaDPO Jawade
Bhavinjawade · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
ML
mlfoundations dev oh v1.3 slim orca x.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
DA
DavidAU MN Dark Planet TITAN 12B
Davidau · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
ML
mlfoundations dev oh v1.3 evol instruct x8
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
dolphin-2.9.2-qwen2-7b
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
BL
v000000 L3 8B BlueSerpentine
Blueserpentine · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
UN
uniwiz-7B-v0.1
Uniwiz · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Agnuxo Llama 3.1 Minitron 4B Instruct CODE Python Spanish English Asistant 16bit v2
Llama · RichardErkhov
Params4.54B
Context128K
Q4_K_M2.8 GB
Coding
LL
MMed Llama 3 8B EnIns
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
TO
tongliuphysics zephyr 7b ultra p 0.07
Tongliuphysics · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TU
Tulu v2.5 Dpo 13b Uf Mean
Tulu · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
FR
Fredithefish Guanaco 3B Uncensored
Fredithefish · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
SU
suehyunpark potpourri 8b inst fft induction bc trajectory max10 per task warm start
Suehyunpark · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
TI
TinyLlama-1.1B-Chat-v0.4
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
MI
osanseviero mistral instruct frankenmerge
Mistral · RichardErkhov
Params8.99B
Context32K
Q4_K_M5.4 GB
GGUF
LZ
lzw1008 Emollama chat 7b
Lzw · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
PY
Pythia 1.4b
Pythia · RichardErkhov
Params1.41B
Context2K
Q4_K_M0.9 GB
GGUF
RE
reflect_llm8B_om2-mstlrg300k460kLlm33130k_SftDpo-MgSpsdpIter2T1_b0.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use

Keep browsing