GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
AIML GEEK promptv5 finetuned CodeLlama7b Instruct
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
BL
Bloom 1b1
Bloom · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh v3.1 wo gpteacher
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Finnish NLP llama 7b finnish instruct v0.2
Llama · RichardErkhov
Params7B
Context2K
Q4_K_M4.3 GB
GGUF
LL
Magpie Align Llama 3 8B Magpie Align v0.3
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
posicube Llama2 chat AYB 13B
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
SA
Salamandra 7b
Salamandra · RichardErkhov
Params7.77B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Leo Hessianai 7b P1 Llama Errors P2 GEC
Llama · RichardErkhov
Params6.74B
Context8K
Q4_K_M4.1 GB
GGUF
HJ
Hjgugugjhuhjggg mergekit ties ysreuuq
Hjgugugjhuhjggg · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
YO
Youri 7b
Youri · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
PH
wenlianghuang Phi3 finance Matt
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
JA
Japanese Stablelm 3b 4e1t Instruct
Japanese · RichardErkhov
Params2.8B
Context4K
Q4_K_M1.7 GB
GGUF
LL
Llama3 70B Chinese Chat
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
GP
p208p2002 gpt2 squad nqg hl
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
EE
EEVE Korean 10.8B Instruct v4 Sft
Eeve · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
TR
travel_microsoft_DialoGPT Small
Travel · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
GP
gpt2-large
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
DE
Deepseek Coder 6.7b Chat
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
CodingTool use
OL
akrishnan TOFU olmo 7b ft full 25epochs lr1e 05 seed42 run1
OLMo · RichardErkhov
Params6.89B
Context4K
Q4_K_M4.2 GB
GGUF
LL
AndreyRzhaksinskiy CDS CodeLlama 7b 20241112 pretrain
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
JJhooww Mistral 7B v0.2 Base ptbr
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ZE
Zephyr 7b Beta_Bio Tutor_Dpo
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DA
DaringMaid-20B-V1.1
Daringmaid · RichardErkhov
Params20B
Context4K
Q4_K_M12.0 GB
GGUF
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
MI
mistral-7b-dolphin-sft
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SM
SmolLM 135M
SmolLM · RichardErkhov
Params135M
Context2K
Q4_K_M0.1 GB
GGUF
ST
Starcoderbase 1b
StarCoder · RichardErkhov
Params1.14B
Context8K
Q4_K_M0.8 GB
Coding
MI
laser-dolphin-mixtral-4x7b-dpo
Mistral · RichardErkhov
Params24.2B
Context32K
Q4_K_M14.6 GB
GGUF

Keep browsing