GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MA
MaziyarPanahi TheTop 5x7B Instruct S4 v0.1
Maziyarpanahi · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SM
sft_smollm_135M
SmolLM · RichardErkhov
Params664M
Context2K
Q4_K_M0.4 GB
GGUF
QW
MiniPLM Qwen 500M
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
GGUF
LL
bedio llama 3.2 1b metamath merged v2
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
TI
TIPO 200M
Tipo · RichardErkhov
Params203M
Context2K
Q4_K_M0.1 GB
GGUF
AU
Audreygyj pythia 160m online dpo ground truth lead merge
Audreygyj · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
GGUF
QW
jbaron34 Qwen2.5 0.5b bebop reranker new small
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Embedding
LL
Omartificial Intelligence Space Arabic llama3.1 16bit FT
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
FA
LosslessMegaCoder Falcon 40b Mini
Falcon · RichardErkhov
Params41.3B
Context2K
Q4_K_M25.2 GB
Coding
PH
g ronimo phi 2 OpenHermes 2.5
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
Tool use
SM
SmolLM2-FT-SQL-Context
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
AM
amit techi gita text generation gpt2
Amit · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
LL
jan hq Llama3.1 s instruct 2024 09 10 v2 cp 3000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
self-reflect_ministral8Bit_mg_star-dpo_2ep
Mistral · RichardErkhov
Params8.02B
Context32K
Q4_K_M4.9 GB
Tool use
IT
ItchyChin OrpoLlama 3 8B memorize translate retrain merge all 20240828
Itchychin · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
GH
s1ghhh dissim round6
Ghhh · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
LL
Andron00e YetAnother Open Llama 3B LoRA OpenOrca
Llama · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
LL
Meta-Llama-3-8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
MI
indischepartij MiniCPM 3B OpenHermes 2.5 v2
MiniCPM · RichardErkhov
Params3.01B
Context2K
Q4_K_M2.0 GB
Tool use
AM
AmberSafe
Ambersafe · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Qwen2.5 Math 72B Instruct
Qwen · RichardErkhov
Params72.7B
Context4K
Q4_K_M47.4 GB
Tool use
MI
Delcos airoboros mistral2.2 Starling 11b
Mistral · RichardErkhov
Params11.4B
Context32K
Q4_K_M6.8 GB
GGUF
ME
mergekit community mergekit slerp duaqshp
Mergekit · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
HJ
Hjgugugjhuhjggg mergekit passthrough ngrncgy
Hjgugugjhuhjggg · RichardErkhov
Params8.04B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama3 S Instruct v0.3 Checkpoint 7000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use

Keep browsing