Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · ATOMIKMN
Params1.86B
Context—
Smallest0.9 GB
MI
Mixtral 8x7B Instruct v0.1
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
TY
Typhoon 7b Instruct 02 19 2024
Typhoon · RichardErkhov
Params7.27B
Context32K
Q4_K_M4.4 GB
MI
MistarlingMaid 2x7B Base
Mistarlingmaid · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
QW
Qwen2.5 Coder 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
BR
Breeze 7B 32k Base V1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GL
Glm Edge V 5b
GLM · zai-org
Params4.33B
Context3K
Q4_K_M2.6 GB
LL
Hermes 3 Llama 3.1 70B
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
MI
nbeerbower Gutensuppe mistral nemo 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
KA
Karakuri Lm 7b Apm v0.1
Karakuri · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GL
glm-4-9b-chat-1m
GLM · legraphista
Params9.48B
Context1024K
Smallest4.0 GB
MI
Mistral Large Instruct 2407
Mistral · legraphista
Params123B
Context32K
Smallest26.0 GB
GE
gemma 4 26B A4B it
Gemma · eaddario
Params25.2B
Context256K
Smallest0.1 GB
DE
deepseek-coder-1.3b-chat-and-function-calling
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
ML
mlfoundations dev stackoverflow 5000tasks 1p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Meta-Llama-3-8B-Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LA
Labradorite 13b
Labradorite · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
VI
Vikhrmodels/Vikhr-YandexGPT-5-Lite-8B-it
Vikhrmodels · Vikhrmodels
Params8.04B
Context32K
Q4_K_M4.9 GB
QW
Replete LLM V2.5 Qwen 14b
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
TL
tlphams solar 10.7b merged v0.1
Tlphams · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
MI
Codestral-22B-v0.1
Mistral · SanctumAI
Params22.2B
Context32K
Q4_K_M13.3 GB
QW
Qwen3.5 9B
Qwen · titan0115
Params8.95B
Context256K
Q4_K_M5.6 GB
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
MA
MadKFC MergedCPsyModel
Madkfc · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
FA
Falcon H1 Tiny Coder 90M
Falcon · tiiuae
Params108M
Context256K
Q4_K_M0.1 GB
NO
notHumpback-Myx
Nothumpback · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
LL
scandukuri llama3 8b stargate m1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
Gemma 2b It
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
DI
distilgpt2
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB