GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Dolphin 2.9 Llama3 8b 1m
Llama · RichardErkhov
Params8.03B
Context1024K
Q4_K_M4.9 GB
GGUF
MI
OpenHermes-2.5-Mistral-7B-16k
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
QW
Qwen2.5 0.5B Instruct
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
Tool use
GR
05 6906grapph
Grapph · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MO
MobileLLM 600M
Mobilellm · RichardErkhov
Params640M
Context2K
Q4_K_M0.5 GB
GGUF
LL
MinaMila llama 3 InstBase adult ImpFeat 1ep withsex
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
NI
nimamegh roberta cnn legal
Nimamegh · RichardErkhov
Params354M
Context512
Q4_K_M0.2 GB
GGUF
GE
gemma-2b-orpo
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
Internlm2 Math 20b Llama
Llama · RichardErkhov
Params19.9B
Context8K
Q4_K_M12.0 GB
GGUF
YI
CaPlatTessDolXaBoros Yi 34B 200K DARE Ties
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
MI
mkebiclioglu mistralai Code Instruct Finetune sunday night test 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
ML
mlfoundations dev oh v1 w v3 opengpt
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
RE
reflect_llm8B_llmMstlrg-om2-80k-SftT2_MgSpsdpT02_b0.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
IN
inflatebot MN 12B Mag Mell R1
Inflatebot · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Reasoning
XU
XueyingJia pythia 1b stage1 dpo SG merge
Xueyingjia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
RL
Rlhflow_Mixture_Iter2
Rlhflow · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
KaKee llama 2 7b full unknown various answer 7 stereo epoch1
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
TI
TinyLlama-1.1B-Chat-v0.2
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
hardikg2907 code llama html completion 1
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
BE
besimray miner id 3 72df7d62 e0d6 41b2 9153 9843320d6b82 1729802122
Besimray · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
QW
Qwen2 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
MI
K2S3 Mistral 7b v1.43
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
MI
WebraftAI synapsellm 7b mistral v0.4 preview3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ML
mlfoundations dev oh v1.3 alpaca x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
XT
xTower13B
Xtower · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
Mistral 7B v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
TinyLlama TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF

Keep browsing