Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
DE
Husky v1 Math Deepseekmath 7b Instruct
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
LL
chinese llama 2 7b
Llama · hfl
Params6.93B
Context4K
Smallest2.6 GB
GGUF
MI
nbeerbower bophades mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RA
RakutenAI 7B chat
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
MI
timestral cpp
Mistral · mudler
Params123M
Context32K
Smallest0.0 GB
GGUF
CH
ChaoticNeutrals Prima LelantaclesV5 7b
Chaoticneutrals · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 Swallow 70B v0.1
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
SP
SparseLlama 3 8B Pruned_50.2Of4
Sparsellama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
PO
Poro 34B
Poro · RichardErkhov
Params34B
Context7K
Q4_K_M22.4 GB
GGUF
QW
Qwen2 VL 2B Instruct
Qwen · turingevo
Params1.54B
Context32K
Q4_K_M1.0 GB
Vision
SM
SmallThinker 21BA3B Instruct
Smallthinker · Tiiny
Params21.5B
Context16K
Q4_K_M13.1 GB
GGUF
SM
gguf SmolLM 135M Instruct
SmolLM · Felladrin
Params135M
Context2K
Smallest0.1 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · SanctumAI
Params3.82B
Context128K
Q4_K_M2.4 GB
Coding
ME
MetaMath Bagel DPO 34B
Metamath · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
HA
Haas-ja-v1.0
Haas · sakasegawa
Params616M
Context—
Smallest0.4 GB
GGUF
OL
OLMo 7B Instruct Hf
OLMo · RichardErkhov
Params6.89B
Context2K
Q4_K_M4.2 GB
GGUF
VI
vinallama-2.7b
Vinallama · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.7 GB
GGUF
CY
z image cyberrealistic v8
Cyberrealistic · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
GGUF
MI
Ministral 3 14B Instruct 2512
Mistral · eaddario
Params13.5B
Context256K
Q4_K_M8.2 GB
Tool use
LL
Meta-Llama-3-8B-Instruct
Llama · SanctumAI
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
gemma 4 31b it assistant mtp
Gemma · NotMe404
Params470M
Context256K
Q4_K_M0.4 GB
GGUF
MM
mmq test
Mmq · Beinsezii
Params14.8B
Context128K
Smallest0.4 GB
ReasoningTool use
LL
Reflection Llama 3.1 70B IMat
Llama · legraphista
Params70.6B
Context8K
Smallest15.3 GB
GGUF
MI
mistral-7b-v0.1-layla-v4-chatml
Mistral · l3utterfly
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
YI
Yi 1.5 6B
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
QW
Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive
Qwen · Unrestricted
Params122B
Context256K
Q4_K_M74.2 GB
ReasoningTool useVision
GE
gemma 4 E4B it
Gemma · eaddario
Params7.52B
Context128K
Smallest0.0 GB
GGUF
VA
VAGOsolutions SauerkrautLM 7b HerO
Vagosolutions · RichardErkhov
Params3.8B
Context—
Q4_K_M2.2 GB
GGUF
LL
microsoft LLaMA 2 7b GTL Delta
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF

Other quants

Keep browsing