Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
GL
glm-4-9b-chat
GLM · legraphista
Params9.4B
Context128K
Smallest3.1 GB
RW
rwkv 6 finch 14b
Rwkv · latestissue
Params14.1B
Context1024K
Smallest5.4 GB
YU
Yuan2 M32 hf IMat
Yuan · legraphista
Params40.2B
Context4K
Smallest8.7 GB
SW
Swallow MS 7b Instruct v0.1
Swallow · RichardErkhov
Params7.33B
Context4K
Q4_K_M4.4 GB
LL
Llama-3-8B-Instruct-MopeyMule
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
BR
Breeze 7B 32k Instruct V1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
Chinese Mixtral 8x7B
Mistral · RichardErkhov
Params46.9B
Context32K
Q4_K_M28.6 GB
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
Granite 3b Code Instruct 128k
Granite · RichardErkhov
Params3.48B
Context125K
Q4_K_M2.1 GB
FA
Falcon H1 7B Instruct
Falcon · tiiuae
Params7.59B
Context256K
Q4_K_M4.6 GB
SN
SniffyOtter 7B
Sniffyotter · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
saraprice llama2 7B backdoor DEPLOYMENT
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
FL
FLUX.2 klein 4B
Flux · adithya-balaji
Params3.88B
Context—
Smallest1.6 GB
DE
DeepSeek-Coder-V2-Instruct
DeepSeek · legraphista
Params236B
Context160K
Smallest47.4 GB
QV
QVikhr 3 4B Instruction
Qvikhr · Vikhrmodels
Params4B
Context40K
Q4_K_M2.5 GB
QW
Qwen2-7B-Instruct
Qwen · legraphista
Params7.62B
Context32K
Smallest1.9 GB
OP
OpenCodeInterpreter-DS-1.3B
Opencodeinterpreter · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
RA
RakutenAI-7B-instruct
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
SW
Swallow 70b Instruct v0.1
Swallow · RichardErkhov
Params69B
Context4K
Q4_K_M41.5 GB
TU
Tulu2 7b_writing Prompts
Tulu · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
rana shahroz mistral 7b openorca20k
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
IM
z image solordz v2
Image · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
GR
Granite 8b Code Base 4k
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
MI
Chinese Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.5B
Context32K
Q4_K_M4.5 GB
CH
chujiezheng Starling LM 7B beta ExPO
Chujiezheng · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
Hermes 3 Llama 3.1 70B
Llama · legraphista
Params70.6B
Context128K
Smallest15.3 GB
RH
rho-math-1b-interpreter-v0.1
Rho · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
MI
UCLA AGI Mistral7B PairRM SPPO Iter3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 32B Instruct Abliterated
Qwen · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB