GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
GR
Granite 3.0 1b A400M Instruct
Granite · RichardErkhov
Params1.33B
Context4K
Q4_K_M0.8 GB
Tool use
VI
vicuna-68m
Vicuna · RichardErkhov
Params68M
Context2K
Q4_K_M0.0 GB
GGUF
ZE
Zephyr 7b Beta
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RE
recogna nlp phibode 3 mini 4k ultraalpaca
Recogna · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
DR TEXT2SQL CodeLlama2 7B
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
KA
kaist ai janus 7b
Kaist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SO
Solar-Ko-Recovery-11B
Solar · RichardErkhov
Params11B
Context4K
Q4_K_M6.6 GB
GGUF
MI
mlfoundations dev mistral alpaca sft sample 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
CPO SimPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
OR
Yi Ko 6B Orca Alpaca Gpt4 Math Lora DPO
Orca · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
LL
Llama3 8B Chinese Chat
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
AI ML Research Qwen 2 0.5B finetuned on 23 september
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
LA
Lancelot53 llm discriminator
Lancelot · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
QW
Internlm2 Math Plus 1_8b
Qwen · RichardErkhov
Params1.89B
Context8K
Q4_K_M1.2 GB
GGUF
LL
CodeLlama 70b Python Hf
Llama · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
Coding
QW
SuperCorrect 7B
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
Tool use
LL
MinaMila llama 3 Base adult ImpFeat 5ep withsex
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
BL
Bloom 560m
Bloom · RichardErkhov
Params816M
Context1K
Q4_K_M0.6 GB
GGUF
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
QW
TechxGenus CursorCore QW2.5 1.5B
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
ML
mlfoundations dev oh teknium scaling down random 1.0
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
zelk12 MT5 Gen1 gemma 2 9B
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
OG
Ogno Monarch Jaskier Merge 7b OH PREF DPO
Ogno · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
TowerInstruct Mistral 7B v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
LL
Vikhr-Llama-3.2-1B-instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
EU
euclaise Memphis CoT 3B
Euclaise · RichardErkhov
Params2.8B
Context4K
Q4_K_M1.7 GB
GGUF
TI
TinyLlama-1.1B-2.5T-chat-and-function-calling
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Tool use
DE
Srijith rkr deepseek base 1e 4 NO cot only failed samples 300 ckpt
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
ST
rm_1014_20K_70K_step_400
Step · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use

Keep browsing