Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
HE
Heejindo rationale model e10 save5000
Heejindo · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
QW
Huihui Qwen3.8 27B abliterated Q6 K
Qwen · ajaeger74
Params27.3B
Context256K
Smallest19.5 GB
ReasoningTool useVision
CH
chinese alpaca 2 13b
Chinese · hfl
Params13.3B
Context4K
Smallest5.0 GB
GGUF
PR
prometheus-7b-v2.0
Prometheus · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
EN
Envoid Yousei 22B
Envoid · RichardErkhov
Params21.8B
Context4K
Q4_K_M13.2 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Svenuks
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
LL
aifeifei798 llama3 8B DarkIdol 2.3 Uncensored 32K
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
ML
mlfoundations dev OH DCFT V3 wo unreplicated
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
TE
tenyx TenyxChat 8x7B v1
Tenyx · RichardErkhov
Params56B
Context32K
Q4_K_M28.4 GB
GGUF
IN
interview eval zephyr 7b stem test
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
mlfoundations dev llama2 oh teknium scaling down random 0.3
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
TI
TIPO 100M
Tipo · RichardErkhov
Params100M
Context8K
Q4_K_M0.1 GB
GGUF
LL
ADT7 SN11 hermes llama3 v6
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Fsfrm_Llama2_Sft_Ver2_Iter1_Dpo
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
CodeLlama 13b Python Hf
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
WA
wamegabe tinyllama wame 4bit curi
Wamegabe · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Tool use
LL
Llama-3-8b-64k-PoSE
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
DA
Darkknight535 OpenCrystal 12B L3
Darkknight · RichardErkhov
Params11.5B
Context8K
Q4_K_M7.0 GB
GGUF
LU
XL LuaCopilot 1.7B FFT
Luacopilot · nwdxlgzs
Params1.72B
Context40K
Q4_K_M1.1 GB
GGUF
LL
CodeLlama 7b Instruct Hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
ML
mlfoundations dev oh teknium scaling down random 0.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
antonytm codellama 34b bnb 4bit sitecore so
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
HE
Heejindo rationale model e3 save5000 f4
Heejindo · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
Llama3 8b Math Sft
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Dolphin 3.0 Llama 3.1 8B
Llama · alex77coder
Params8.03B
Context128K
Q4_K_M4.9 GB
CodingTool use
LL
vidhirambhia test llama math coder
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Coding
QW
robinsmits Qwen1.5 7B Dutch Chat
Qwen · RichardErkhov
Params7.72B
Context32K
Q4_K_M4.8 GB
GGUF
AZ
Azazelle Half NSFW Noromaid 7b
Azazelle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
ndebuhr Mistral 7B Technical Tutorial Summarization QLoRA
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Papy_1_Llama 3.1 8B Instruct_date
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use

Other quants

Keep browsing