Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama-2-7b-dolphin-open_platypus-pruned_70
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GP
Gpt2Finetuned
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
TI
TinyAlpaca-1.1B
Tinyalpaca · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
RJ1200 llama 31b question
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
ML
mlfoundations dev oh v1 2 only metamath
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
OpenPipe mistral ft optimized 1218
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
TinyLlama-1.1B-intermediate-step-480k-1T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LG
lgaalves gpt2 platypus dolly guanaco
Lgaalves · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
OP
OpenELM 3B SFT D_chosen HuggingFaceH4 Ultrafeedback_Binarized Xlarge
Openelm · RichardErkhov
Params3.04B
Context2K
Q4_K_M1.9 GB
GGUF
LL
llama2-7b-hf-guanaco
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama 3.1 TAIDE LX 8B Chat
Llama · nctu6
Params8.52B
Context128K
Q4_K_M5.3 GB
Tool use
HA
Hastagaras L3.2 JametMini 3B MK.III
Hastagaras · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
GE
nooynoos gemma2 2b M.O.M gemma sprint
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
AN
anthonymeo full train
Anthonymeo · RichardErkhov
Params1.89B
Context32K
Q4_K_M1.2 GB
GGUF
KA
karthikeyan r calculation model 6k
Karthikeyan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
RE
RedPajama-INCITE-Chat-3B-Instruction-Tuning-with-GPT-4
Redpajama · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
AE
aetherwiing MN 12B Starcannon v2
Aetherwiing · RichardErkhov
Params12.2B
Context1000K
Q4_K_M3.4 GB
GGUF
LL
omniquad del llama3.2 text sql fp16 merged v1.1
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
ON
OneReward
Onereward · befox
Params11.9B
Context—
Smallest4.0 GB
GGUF
LL
OpenRLHF Llama 2 7b sft model ocra 500k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
CodeActAgent Llama 2 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
CodeLlama 13B Fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
BR
broalantap GPT2 large 4 16000steps
Broalantap · RichardErkhov
Params839M
Context1K
Q4_K_M0.5 GB
GGUF
TR
Tristan dclm fasttext oh eli5 1b
Tristan · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
MI
mlfoundations dev mistral alpaca sft sample 8
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
llama-3-8B-Instruct-function-calling-v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
GE
christinechagas gemma 2b instruct ft health bot
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
sanchit gandhi distil mistral 1.5B v0.1
Mistral · RichardErkhov
Params1.57B
Context32K
Q4_K_M1.0 GB
GGUF
QW
masa research Qwen2 0.5B 20240829 175401
Qwen · RichardErkhov
Params494M
Context128K
Q4_K_M0.4 GB
GGUF
XA
xammi final mp model merged
Xammi · RichardErkhov
Params1.1B
Context4K
Q4_K_M0.7 GB
GGUF

Other quants

Keep browsing