GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
GE
nathsay23 gemma 2b Science qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
QW
Qwen2 0.5B Instruct
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
LL
freddy016 merged model sentiment llama3
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
PH
bunnycore Phi 3.5 Mini Sonet RP
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
Roleplay
WE
Weyaxi MetaMath Tulpar 7b v2 Slerp
Weyaxi · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SM
SmolLM2-FT-wikihow_es_v2
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
LU
LunaticMaestro gpt stories
Lunaticmaestro · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
AT
Atlas Chat 27B
Atlas · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
FA
failspy kappa 3 phi abliterated
Failspy · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
Llama-3-8B-Instruct-Gradient-4194k
Llama · RichardErkhov
Params8.03B
Context1K
Q4_K_M4.9 GB
GGUF
NE
Nekomata 7b Instruction
Nekomata · RichardErkhov
Params7.72B
Context32K
Q4_K_M4.9 GB
GGUF
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
QW
Qwen2-0.5B-Chat_SFT_LoRA
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
RE
realPCH ko solra platusv3 koprompt
Realpch · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
GGUF
SP
spanish-dialoGPT
Spanish · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.15
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
JA
japanese-stablelm-instruct-beta-70b
Japanese · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
LL
Meta Llama 3 8B Instruct Function Calling Json Mode
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10 minlr5e 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2b It SFT D1_chosen HuggingFaceH4 Ultrafeedback_Binarized Xlarge
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
EN
ENERGY DRINK LOVE leaderboard inst v1.3 Open Hermes LDCC SOLAR 10.7B DPOv3
Energy · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
Tool use
NO
Nondzu zephyr speakleash 010 pl 3072 32 16 0.01
Nondzu · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
vivaming Llama 3.2 3B 16bit factify liar vllm
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
LL
Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SI
silma ai SILMA 9B Instruct v0.1.1
Silma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
GE
Gemma 2 2b It General1.2M Trc313Eval45
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GE
unsloth gemma 7b it
Gemma · RichardErkhov
Params7B
Context—
Q4_K_M3.7 GB
GGUF
LL
H2Ogpt 32k Codellama 34b Instruct
Llama · RichardErkhov
Params33.7B
Context32K
Q4_K_M20.2 GB
Coding

Keep browsing