Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
DsnTgr llama 3.2 3b it Ecommerce ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
AL
ALMA-13B-R
Alma · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GE
gemma-2b-it-tamil-v0.1-alpha
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
MA
Manticore 30b Chat Pyg Alpha
Manticore · RichardErkhov
Params32.5B
Context2K
Q4_K_M19.6 GB
MI
thkim0305 Mistral 7B ours algo7s lyr20 n11 0.5 0.5 0.1 0.1 200steps full
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ME
Medicine Chat
Medicine · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
lamm mit meta llama Llama 3.2 3B Instruct untied
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
MI
OpenPipe mistral ft optimized 1227
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5-0.5B-SFT-2e-5-5ep
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
MI
mlfoundations dev hp ablations grid mistral bsz4096 lr2e 6 scheduler cosine warmup0.15
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
zebraLLAMA zebra Llama v0.2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DE
ahmedheakl asm2asm deepseek 1.3b 100k float16 attn
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
DO
DopeorNope You can cry Snowman 13B
Dopeornope · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.0 GB
GR
Granite 3.0 1b A400M Instruct
Granite · RichardErkhov
Params1.33B
Context4K
Q4_K_M0.8 GB
VI
vicuna-68m
Vicuna · RichardErkhov
Params68M
Context2K
Q4_K_M0.0 GB
ZE
Zephyr 7b Beta
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RE
recogna nlp phibode 3 mini 4k ultraalpaca
Recogna · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
LL
DR TEXT2SQL CodeLlama2 7B
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
KA
kaist ai janus 7b
Kaist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SO
Solar-Ko-Recovery-11B
Solar · RichardErkhov
Params11B
Context4K
Q4_K_M6.6 GB
MI
mlfoundations dev mistral alpaca sft sample 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
CPO SimPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
OR
Yi Ko 6B Orca Alpaca Gpt4 Math Lora DPO
Orca · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
LL
Llama3 8B Chinese Chat
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
AI ML Research Qwen 2 0.5B finetuned on 23 september
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
LA
Lancelot53 llm discriminator
Lancelot · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Riyan200324200324
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Internlm2 Math Plus 1_8b
Qwen · RichardErkhov
Params1.89B
Context8K
Q4_K_M1.2 GB
LL
CodeLlama 70b Python Hf
Llama · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
QW
SuperCorrect 7B
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB