GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2-7B-Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Orenguteng Llama 3 8B LexiFun Uncensored V1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 13b
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
MI
NousResearch Nous Hermes 2 Mixtral 8x7B DPO
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
Tool use
LL
nota ai st llama 1 5.5b ppl
Llama · RichardErkhov
Params5.52B
Context2K
Q4_K_M3.3 GB
GGUF
BR
BroAlanTaps GPT2 large 128 10000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
RE
realPCH kosolra wiki QA
Realpch · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
GGUF
LL
llama3-8b-math-sft-mix-2-0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
jpalgo llama 3.1 sl12k overwrite v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
Qwen2.5 Coder 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
YI
Yi 1.5 9B Chat
Yi · RichardErkhov
Params8.83B
Context8K
Q4_K_M5.3 GB
GGUF
ST
StarCoder
StarCoder · RichardErkhov
Params1.14B
Context8K
Q4_K_M0.8 GB
Coding
PH
Phi 3.5 Mini IFRS Accountant
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
AK
akshit Gupta smol360M fullnew merged FinetunedByAG
Akshit · RichardErkhov
Params362M
Context2K
Q4_K_M0.3 GB
GGUF
FP
FPHam L3 8B Everything COT
Fpham · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GP
sp8108 gpt2 finetuned ros2 v3
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
QW
trl internal testing tiny Qwen2ForCausalLM 2.5 Coder
Qwen · RichardErkhov
Params11M
Context—
Q4_K_M0.0 GB
Coding
WE
Wenzhong-GPT2-110M
Wenzhong · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
ML
mlfoundations dev oh dcft v1.1 no curation
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
OP
OpenELM 1_1B IPO
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Meta LLama 3 Cat A LLama 70b
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
MI
cognitivecomputations dolphin 2.6 mistral 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Suzume Llama 3 8B Multilingual
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
KA
Katayoon DPO ONLY Zephyr 7B 3 iter 2
Katayoon · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
VO
vonPipe selfiesPretrainInstruct
Vonpipe · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SO
SOLAR 10.7B Instruct STOCK_SOLAR Slerp
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
zomd AISquare Instruct llama2 koen 13b v0.9.19
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Keep browsing