Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
zomd AISquare Instruct llama2 koen 13b v0.9.19
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Vbd Llama2 7B 50b Chat
Llama · RichardErkhov
Params6.88B
Context4K
Q4_K_M4.2 GB
MI
mlfoundations dev hp ablations mistral epoch2 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mlfoundations dev hp ablations mistral bsz1024
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DA
Dans Archive Dans MysteryModel 13b
Dans · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
LL
jamesohe Llama3 CAS Audit8B GCNI V3
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
Mistral Mcqa Circuit Broken
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ML
mlfoundations dev oh v1.3 camel chemistry x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
Gemma 2b It SFT D1_chosen Orca
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
Llama 3.2 1B Instruct De
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
NI
NIyengar doct qna finetune gpt2
Niyengar · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
RE
reflect_llm8B_Om2SftT2_Om2G8kOm2AgBlcIpsdpIter1T02_b0.1
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2 0.5B Instruct
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
MI
DeepKarkhanis Mistral Passthrough 8L 10B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Neural-grok-dolphin-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
StarCoder
StarCoder · RichardErkhov
Params164M
Context8K
Q4_K_M0.1 GB
PH
Phi 3 Mini 128k Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
HA
harkov000 hermes dolphin 8B
Harkov · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10 minlr5e 7 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mlfoundations dev hp ablations mistral bsz1024 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
CH
ChaoticNeutrals Poppy Porpoise v0.6 L3 8B
Chaoticneutrals · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
sonthenguyen NeuralHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 3b It Ecommerce ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
JO
JOSIEv4o 8b Stage1 Beta1
Josiev · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
IL
ilsp Meltemi 7B v1
Ilsp · RichardErkhov
Params7.48B
Context32K
Q4_K_M4.5 GB
GE
Gemma 2 Baku 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
QW
leniachat-qwen2-1.5B-v0
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
LL
abhinand Llama 3 Galen 8B 32k v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB