Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
NE
NeuralExperiment 7b MagicCoder v7
Neuralexperiment · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
unsloth Hermes 2 Pro Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TI
TinyLlama 1.1B Tele
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
RU
ruchi012 saul 7B merged 3000
Ruchi · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LU
lumatic ai BongLlama 1.1B Chat alpha v0
Lumatic · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
MI
Mistral-7B-Merge-14-v0.5
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
Phi 3 Medium 4k Instruct
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.4 GB
NO
franken SOLAR 18B v1.0
Nous · vicgalle
Params17.9B
Context4K
Smallest6.6 GB
MI
NeuralHermes-2.5-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
jan hq llama3 s instruct v0.3 checkpoint 6000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
JINJIN7987 llama3 3b neg mtba
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · ATOMIKMN
Params7.52B
Context128K
Q4_K_M5.3 GB
LL
popatkishan1 Llama 3 8B Instruct AIH RAG
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
ZU
a1zubair RedditData Fine Tuned GPT2
Zubair · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
LL
Dolphin 2.9 Llama3 8b 1m
Llama · RichardErkhov
Params8.03B
Context1024K
Q4_K_M4.9 GB
MI
OpenHermes-2.5-Mistral-7B-16k
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
QW
Qwen2.5 0.5B Instruct
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
GE
Gemma-4-E2B-Uncensored-HauhauCS-Aggressive
Gemma · Unrestricted
Params4.65B
Context128K
Smallest3.0 GB
GR
05 6906grapph
Grapph · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
CharGen v2
Mistral · kubernetes-bad
Params7.24B
Context32K
Q4_K_M4.4 GB
MO
MobileLLM 600M
Mobilellm · RichardErkhov
Params640M
Context2K
Q4_K_M0.5 GB
LL
MinaMila llama 3 InstBase adult ImpFeat 1ep withsex
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
NI
nimamegh roberta cnn legal
Nimamegh · RichardErkhov
Params354M
Context512
Q4_K_M0.2 GB
GE
gemma-2b-orpo
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
Internlm2 Math 20b Llama
Llama · RichardErkhov
Params19.9B
Context8K
Q4_K_M12.0 GB
YI
CaPlatTessDolXaBoros Yi 34B 200K DARE Ties
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
MI
mkebiclioglu mistralai Code Instruct Finetune sunday night test 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ML
mlfoundations dev oh v1 w v3 opengpt
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
RE
reflect_llm8B_llmMstlrg-om2-80k-SftT2_MgSpsdpT02_b0.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB