Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
Kabster Bio Mistralv2 Squared
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ZE
zephyr-7b-dpo-full-prompt-extend-chosen
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
irlab udc Llama 3.1 8B Instruct Galician
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DE
deepnet C00ReadyModel
Deepnet · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
gemma-2b-ForexAI
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
Hermes 2 Theta Llama 3 70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
LL
Dolphin 3.0 Llama 3.1 8B
Llama · R4xx1
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · jchonde
Params1.86B
Context256K
Smallest0.9 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
OA
h2oai h2o danube2 1.8b sft
Oai · RichardErkhov
Params1.83B
Context8K
Q4_K_M1.1 GB
IA
iamspathan half girlfriend gpt2
Iamspathan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
MI
s3nh nsfw noromaid mistral instruct
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SH
Shieldgemma 2b
Shieldgemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
DP
Yi Ko 6B DPO v2
Dpo · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
MI
Mixtral_7Bx2_MoE_13B
Mistral · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
NO
nothingiisreal L3 8B Celeste v1
Nothingiisreal · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
AL
allknowingroger MultiverseEx26 Neurallaymons 12B MoE
Allknowingroger · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GE
yhkim9362 gemma en ko 7b v0.1
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
DE
yiran wang3 clean mining sft split deepseek packing sft
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
AS
asiansoul SmartLlama 3 Ko 8B 256k PoSE
Asiansoul · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
thusinh1969 Guardian V0.1 LLaMA3.1 8B 5G 6Oct2024 epoch1.3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
AXCXEPT EZO Humanities 9B gemma 2 it
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
SA
saucam Arithmo Wizard 2 7B
Saucam · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
IN
InternLM2
InternLM · RichardErkhov
Params1.89B
Context32K
Q4_K_M1.2 GB
MI
Mistral 7B v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
VI
Vidyayini gita text generation gpt2
Vidyayini · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
MI
openinstruct-mistral-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
OP
OpenELM 1_1B DPO Full 2 5
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB