Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
Bitnet Mistral.0.2 47m v0.2 Grokfast1
Mistral · RichardErkhov
Params330M
Context4K
Q4_K_M0.2 GB
ST
Starcoderbase 1b
StarCoder · RichardErkhov
Params1.24B
Context8K
Q4_K_M0.8 GB
QW
Agnuxo Qwen2 1.5B Instruct MOE CODE assistant 16bit
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
QW
Qwen2.5 32B Instruct
Qwen · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.8 GB
JP
jpalgo MARK 3000 QLORA v2
Jpalgo · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
OL
OLMo 7B 0724 Instruct Hf
OLMo · RichardErkhov
Params6.89B
Context2K
Q4_K_M4.2 GB
LL
Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GE
GeneZC MiniChat 1.5 3B
Genezc · RichardErkhov
Params3.02B
Context4K
Q4_K_M1.8 GB
LL
codellama CodeLlama 34b hf
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
OP
openchat-3.5-0106-11b
Openchat · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
LL
Llama 3 Youko 70b Instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
SY
SystemGemma2 9b It
Systemgemma · RichardErkhov
Params9.24B
Context32K
Q4_K_M5.8 GB
EN
ennodata-13b-8bit-raw-15epoch
Ennodata · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GP
Gpt2 Xl
Gpt · RichardErkhov
Params1.64B
Context1K
Q4_K_M1.2 GB
GP
gpt2-medium-conversational
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
LL
Llama 3.2 1B Instruct Ja Gsm8K Layerswap
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
ME
MentaLLaMA-chat-7B-hf
Mentallama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
AL
allknowingroger MultiverseMath 12B MoE
Allknowingroger · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
ML
mlfoundations dev oh teknium scaling down ratiocontrolled 0.9
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama-3.2-3B-Instruct-cot-19-11-2
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
DE
KSU HW SEC Deepseek SFT RL Text DFG AST
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
llama-2-ko-7b-emb-dev
Llama · RichardErkhov
Params6.86B
Context2K
Q4_K_M4.2 GB
QW
Qwen3.6-27B-Uncensored-HauhauCS-Aggressive
Qwen · Unrestricted
Params26.9B
Context256K
Smallest10.0 GB
ML
mlfoundations dev oh v1.3 evol instruct x2
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama 3 8B Instruct 80K QLoRA Merged
Llama · Monor
Params8.03B
Context80K
Q4_K_M4.9 GB
GP
Gpt2_Large_Tdata1_Fdata_30_Epochs
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GP
Gpt Neox 20b Embeddings
Gpt · RichardErkhov
Params20.6B
Context2K
Q4_K_M13.1 GB
JP
jpalgo EMP FT 3K V1
Jpalgo · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB