Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
FrancescoBuda Llama ICD coder 3B merged 1ep
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
CodingTool use
MI
K2S3 Mistral 7b v1.47
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Qwen 2.5 3b Evol CoT
Qwen · RichardErkhov
Params3.4B
Context32K
Q4_K_M2.1 GB
Tool use
MI
Mistral Instruct DPO Y24 v2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PY
Pythia 1b Sft Hh
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
MI
Locutusque Hyperion 2.1 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr1e 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
PH
Phi 3 Mini 4k Instruct Dpo
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
AL
AlphaMist7B Slr v4 Slow
Alphamist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
HO
Holarissun gpt2 sft tldr
Holarissun · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
mlfoundations dev hp ablations mistral lr2e 6 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
microsoft Phi 3 medium 4k instruct
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.6 GB
GGUF
GI
Giraffe-v2-70b-32k
Giraffe · RichardErkhov
Params69B
Context32K
Q4_K_M41.4 GB
GGUF
DE
Srijith rkr deepseek base 1e 3 NO cot all samples 300 ckpt
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
LL
akshit Gupta llamanew1merged FinetunedByAG
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
CR
crestf411 L3.1 8B sunfall v0.6.1 dpo
Crestf · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
jan hq llama3 s instruct v0.3 checkpoint 3000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama3 S Instruct v0.3 Checkpoint 7000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
ON
Only_Mini
Only · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
PY
Pythia 1b
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
YO
yoshility model merge dare ties v1
Yoshility · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GR
GritLM 7B
Gritlm · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
lmlab-mistral-1b-untrained
Mistral · RichardErkhov
Params1.13B
Context32K
Q4_K_M0.7 GB
GGUF
LL
Llama3 8B Dpo Sft v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
YU
yuntian deng gpt2 implicit cot multiplication 20 digits
Yuntian · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF

Other quants

Keep browsing