Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
AV
average-dolphin-8x7B
Average · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
PH
microsoft Phi 3 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
TO
tokyotech llm Swallow 13b instruct hf
Tokyotech · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
NU
NuminaMath-7B-TIR
Numinamath · legraphista
Params6.91B
Context4K
Smallest1.7 GB
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · Amory0201
Params7.5B
Context128K
Q4_K_M5.3 GB
LL
Llama 2 13b Hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
RW
rwkv 6 finch 1b6
Rwkv · latestissue
Params1.6B
Context1024K
Smallest0.7 GB
LL
Llama3 8b Cpt Sea Lionv2 Instruct
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen3-ASR-0.6B
Qwen · sakasegawa
Params782M
Context—
Smallest0.5 GB
UC
UCLA AGI zephyr 7b sft full SPIN iter3
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
realtreetune deepseekmath 7b sft GSM8K
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GR
Granite 34b Code Base 8k
Granite · RichardErkhov
Params33.7B
Context8K
Q4_K_M21.4 GB
MI
K2S3 Mistral 7b v1.46
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
MI
Mistral-7B-v0.3
Mistral · legraphista
Params7.25B
Context32K
Smallest1.6 GB
GE
Gemma 2 2b
Gemma · legraphista
Params2.61B
Context8K
Smallest0.8 GB
ST
stabilityai japanese stablelm instruct gamma 7b
Stabilityai · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GI
Gille StrangeMerges 50 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Emma 500 Llama2 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LO
louisbrulenaudet Maxine 7B 0401 stock
Louisbrulenaudet · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DR
DreadPoor Harpy 7B Model Stock
Dreadpoor · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
UC
UCLA AGI zephyr 7b sft full SPIN iter2
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
NI
NicholasCorrado zephyr 7b uf rlced conifer dpo 2e
Nicholascorrado · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Minitron 4B Base
Minitron · legraphista
Params4.19B
Context4K
Smallest1.9 GB
LL
cognitivecomputations dolphin 2.9 llama3 8b
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
MI
Minimax H3 pruned
Minimax · PurpleBlazey
Params20.1B
Context256K
Smallest6.3 GB
VI
vicgalle Configurable Janus 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 14B Instruct Abliterated v2
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
MI
failspy Codestral 22B v0.1 abliterated v3
Mistral · RichardErkhov
Params22.2B
Context32K
Q4_K_M13.3 GB
MI
AmeerH Mistral Math 2x7b mix
Mistral · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GE
Gemma 2 27b It Abliterated
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB