Q5_K_M GGUF models
5-bit k-quant, medium. A step up in quality from Q4_K_M for a modest size increase. 117156 models have a Q5_K_M file. Sizes run from 1M to 1570B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3 1.7B
Qwen · gaianet
Params2.03B
Context40K
Q4_K_M1.3 GB
ST
stable code 3b
Stable · TheBloke
Params2.8B
Context16K
Q4_K_M1.7 GB
OL
OLMo 2 1124 7B Instruct
OLMo · bartowski
Params7.3B
Context4K
Q4_K_M4.5 GB
JA
Jan-v3-4b-base-instruct
Jan · janhq
Params4.4B
Context256K
Q4_K_M2.7 GB
WE
WeVisOne VL 2B
Wevisone · mradermacher
Params2.03B
Context256K
Q4_K_M1.3 GB
GR
Granite 4.2 3b
Granite · bartowski
Params3.66B
Context128K
Q4_K_M2.3 GB
HE
Hermes 4.3 36B Heretic
Hermes · mradermacher
Params36.2B
Context512K
Q4_K_M21.8 GB
SE
Seed OSS 36B Instruct Abliterated
Seed · mradermacher
Params36.2B
Context512K
Q4_K_M21.8 GB
MI
WikiHow Mistral Instruct 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
LL
Llama 3.2 3B Instruct Abliterated
Llama · mradermacher
Params3.61B
Context128K
Q4_K_M2.2 GB
RI
Ringg Router E2B
Ringg · bartowski
Params4.63B
Context128K
Q4_K_M3.2 GB
DP
openaccess-ai-collective_dpopenhermes-7b-v2
Dpopenhermes · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma 4 12B Agentic Fable5 Composer2.5 v2 3.5x Tau2 Uncensored Heretic
Gemma · llmfan46
Params11.9B
Context256K
Q4_K_M7.4 GB
AI
Airoboros L2 13B 3.1.1
Airoboros · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LL
Llama-3.1-Nemotron-Nano-8B-V1
Llama · unsloth
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen3.5 9B The Defiant Fable Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · zugquants
Params8.95B
Context256K
Q4_K_M6.8 GB
AI
jondurbin_airoboros-m-7b-3.1.2
Airoboros · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.5 27B Claude 4.6 Opus Reasoning Distilled Heretic
Qwen · mradermacher
Params26.9B
Context256K
Q4_K_M16.5 GB
ME
Med Advisor Conversation 4B
Med · mradermacher
Params4.41B
Context40K
Q4_K_M2.7 GB
ST
L3 8B Stheno v3.1
Stheno · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
TE
Tencent Hy MT2 30B A3B
litigerking
Params30B
Context256K
Q4_K_M18.2 GB
MA
Magnum 12b v2.5 Kto
Magnum · bartowski
Params12.2B
Context1000K
Q4_K_M7.5 GB
GP
GPT OSS Swallow 120B RL v0.1
gpt-oss · mradermacher
Params117B
Context128K
Q4_K_M87.9 GB
LL
Hermes 3 Llama 3.1 405B
Llama · bartowski
Params405B
Context128K
Q4_K_M243.1 GB
JA
Jais adapted 7B Reflection Tuning Natural Farmer i1
Jais · mradermacher
Params7B
Context4K
Q4_K_M4.3 GB
OR
Orion 26B A4B v1.1
Orion · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
GE
Gemma 4 Queen 31B It
Gemma · mradermacher
Params30.7B
Context256K
Q4_K_M18.7 GB
RU
surogate rune 26b a4b
Rune · owao
Params25.2B
Context256K
Q4_K_M16.8 GB
KR
Kroma v0.3.1 Turbo OPD
Kroma · Abiray
Params12.8B
Context—
Q4_K_M8.8 GB
QW
Huihui Qwen3.5 35B A3B Claude 4.6 Opus Abliterated
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB