Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
AV
average-dolphin-8x7B
Average · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
PH
microsoft Phi 3 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
TO
tokyotech llm Swallow 13b instruct hf
Tokyotech · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
GGUF
NU
NuminaMath-7B-TIR
Numinamath · legraphista
Params6.91B
Context4K
Smallest1.7 GB
GGUF
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · Amory0201
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
LL
Llama 2 13b Hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
RW
rwkv 6 finch 1b6
Rwkv · latestissue
Params1.6B
Context1024K
Smallest0.7 GB
GGUF
LL
Llama3 8b Cpt Sea Lionv2 Instruct
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3-ASR-0.6B
Qwen · sakasegawa
Params782M
Context—
Smallest0.5 GB
GGUF
UC
UCLA AGI zephyr 7b sft full SPIN iter3
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
realtreetune deepseekmath 7b sft GSM8K
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
GR
Granite 34b Code Base 8k
Granite · RichardErkhov
Params33.7B
Context8K
Q4_K_M21.4 GB
Coding
MI
K2S3 Mistral 7b v1.46
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
MI
Mistral-7B-v0.3
Mistral · legraphista
Params7.25B
Context32K
Smallest1.6 GB
GGUF
GE
Gemma 2 2b
Gemma · legraphista
Params2.61B
Context8K
Smallest0.8 GB
GGUF
ST
stabilityai japanese stablelm instruct gamma 7b
Stabilityai · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
GI
Gille StrangeMerges 50 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Emma 500 Llama2 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LO
louisbrulenaudet Maxine 7B 0401 stock
Louisbrulenaudet · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DR
DreadPoor Harpy 7B Model Stock
Dreadpoor · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
UC
UCLA AGI zephyr 7b sft full SPIN iter2
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NI
NicholasCorrado zephyr 7b uf rlced conifer dpo 2e
Nicholascorrado · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Minitron 4B Base
Minitron · legraphista
Params4.19B
Context4K
Smallest1.9 GB
GGUF
LL
cognitivecomputations dolphin 2.9 llama3 8b
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Minimax H3 pruned
Minimax · PurpleBlazey
Params20.1B
Context256K
Smallest6.3 GB
GGUF
VI
vicgalle Configurable Janus 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 14B Instruct Abliterated v2
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
GGUF
MI
failspy Codestral 22B v0.1 abliterated v3
Mistral · RichardErkhov
Params22.2B
Context32K
Q4_K_M13.3 GB
Coding
MI
AmeerH Mistral Math 2x7b mix
Mistral · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
GE
Gemma 2 27b It Abliterated
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF

Other quants

Keep browsing