Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
MI
K2S3 Mistral 7b v1.42
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
PH
Phi 3.5 Mini Instruct LoRA 128
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
MI
Hertz Mistral Hermes 2x7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Xennon BD Qwen uncensored v2
Qwen · RichardErkhov
Params494M
Context128K
Q4_K_M0.4 GB
MI
MiniMA 3B
Minima · RichardErkhov
Params3.02B
Context4K
Q4_K_M1.8 GB
PH
M1LL1X microsoft phi 3 5 mini trained abliterated QA 1960 all modules merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
NI
Nitral AI Nyan Stunna 7B
Nitral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GP
Gpt2_Medium_Tdata2_40_Epochs
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
MI
EM German
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
finetune_starcoder2_3b_tokens_added_6k_steps
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
LL
Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · stronman
Params7.52B
Context128K
Q4_K_M5.3 GB
JA
jaspionjader Kosmos EVAA PRP v27 8B
Jaspionjader · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
BR
BroAlanTaps GPT2 large 128 28000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GR
Granite 3.0 3b A800M Instruct
Granite · RichardErkhov
Params3.37B
Context4K
Q4_K_M2.1 GB
LL
jkazdan llama8b gsm real sftsd1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Llama 2 Koen 13b
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
LL
ValiantLabs CodeLlama 13B Fireplace
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
LL
Llama3 S Instruct v0.3 Checkpoint 7000 Phase 3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
IN
InternLM2
InternLM · RichardErkhov
Params7.74B
Context8K
Q4_K_M4.7 GB
SO
SOLAR 10.7B Instruct v1.0 128k
Solar · RichardErkhov
Params10.7B
Context128K
Q4_K_M6.5 GB
KI
Chat Error Kimiko 10.7B v3
Kimi · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
LL
CodeLlama 13B Python Fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
ML
mlfoundations dev oh teknium scaling down ratiocontrolled 0.3
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
ST
Starling LM 7B beta
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
MI
IntelLabs sqft sparsepeft mistral 7b v0.3 60 gsm8k heu
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive
Qwen · botp
Params122B
Context256K
Q4_K_M74.2 GB
LL
smol_llama-4x220M-MoE
Llama · RichardErkhov
Params595M
Context2K
Q4_K_M0.4 GB