Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
OpenBuddy openbuddy llama3.1 8b v22.3 131k
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
CA
CatPPT base
Catppt · andrijdavid
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
incisolr gemma 2b instruct ft datascienceinterview qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
UCLA AGI Llama 3 Instruct 8B SPPO Iter3
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 2 KoEn 13B
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
DE
KSU HW SEC Deepseek SFT RL DFG only
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
LL
PlatYi 34B Llama Q v2
Llama · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
OL
OLMo 7B 0724 Hf
OLMo · aifoundry-org
Params6.89B
Context4K
Smallest1.6 GB
GGUF
FU
FusionNet_7Bx2_MoE_14B
Fusionnet · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
MI
Mistral Class Tutor 7b Ep3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 2 13b Chat Hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
SA
SambaLingo Turkish Chat
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
GGUF
BR
BroAlanTaps GPT2 large 4 34000steps
Broalantaps · RichardErkhov
Params839M
Context1K
Q4_K_M0.5 GB
GGUF
ME
mergekit community mergekit slerp tsuvyfz
Mergekit · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
pucpr br Clinical BR Mistral 7B v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
jesse tong llama 2 7b chat newspaper vi
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
nvidia OpenMath2 Llama3.1 70B
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
QW
Qwen2.5 Math 1.5B
Qwen · RichardErkhov
Params1.54B
Context4K
Q4_K_M1.0 GB
Tool use
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
LL
mlfoundations dev llama2 oh teknium scaling down random 1.0
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
broalantap Llama3 Compressor Adapter 4 20000steps
Llama · RichardErkhov
Params839M
Context1K
Q4_K_M0.5 GB
GGUF
CH
ChemDFM 13B v1.0
Chemdfm · RichardErkhov
Params13.1B
Context2K
Q4_K_M7.9 GB
GGUF
BR
BroAlanTaps GPT2 large 64 15750steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
VI
vilm vinallama 12.5b chat DUS
Vilm · RichardErkhov
Params12.5B
Context4K
Q4_K_M7.6 GB
GGUF
MI
mistral-11b-slimorca
Mistral · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
LL
Llama 3.2 1B Instruct Ja
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
realshyfox sharded Llama 3 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing