Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
hf 100 Llama 3.1 Spellbound StoryWriter 70b instruct 0.3 16bit
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
JI
jina v5 nano
Jina · cstr
Params212M
Context8K
Smallest0.2 GB
Embedding
ST
Cerebras Step 3.5 Flash REAP 149B A11B
Step · wimmmm
Params149B
Context256K
Smallest0.4 GB
GGUF
MI
Voxtral-Mini-4B-Realtime-2602
Mistral · andrijdavid
Params4.43B
Context—
Q4_K_M2.9 GB
GGUF
BE
beomi Yi Ko 34B Chat Preview
Beomi · RichardErkhov
Params34.6B
Context2K
Q4_K_M20.8 GB
GGUF
GL
Glm Edge V 2b
GLM · zai-org
Params1.59B
Context2K
Q4_K_M1.0 GB
Vision
SA
saishf Top Western Maid 7B
Saishf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
QW
Qwen3.5 9B
Qwen · eaddario
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
QW
Qwen3.5 2B Base
Qwen · titan0115
Params1.88B
Context256K
Q4_K_M1.3 GB
ReasoningTool use
DE
Deepseekmath 7b Sft MATH v2
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
GO
Gonzo Code 7B
Gonzo · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
QW
Qwen3.8-27B
Qwen · Aydge
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
GR
Granite 20b Code Instruct 8k
Granite · RichardErkhov
Params20.1B
Context8K
Q4_K_M12.8 GB
Coding
WA
WAN2.2 14B Rapid AllInOne v12.2
Wan · Novice25
Params17.3B
Context—
Smallest6.4 GB
GGUF
QW
Qwen3.8-27B
Qwen · Didkk
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
QW
qwen image edit
Qwen · badgame
Params20.4B
Context—
Smallest11.5 GB
GGUF
LL
llama-3-chinese-8b-instruct-v3
Llama · hfl
Params8.03B
Context8K
Smallest3.2 GB
GGUF
MI
Ft Mistral 7b Instruct v0.2 Sorry Bench 202406
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
lemon mint gemma ko 7b it v0.41
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
QW
Qwen3.5 9B
Qwen · ZuzeTt
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
GL
Gliese Query Tool 0.6B
Gliese · prithivMLmods
Params596M
Context40K
Q4_K_M0.4 GB
ReasoningTool use
SI
Sixtyoneeighty 7b
Sixtyoneeighty · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DR
Dracarys2 72B Instruct
Dracarys · RichardErkhov
Params72.7B
Context32K
Q4_K_M47.4 GB
Tool use
LL
Configurable-Llama-3-8B-v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
ihughes15234 llama 3 1 8bi tictactoe dpo3epoch v2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
VI
Vistral 24B Instruct
Vistral · Vikhrmodels
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
OP
OpenELM-270M
Openelm · RichardErkhov
Params272M
Context2K
Q4_K_M0.2 GB
GGUF
FL
flux.1 lite 8B
Flux · Green-Sky
Params8.16B
Context—
Smallest2.9 GB
GGUF
LL
PatronusAI Llama 3 Patronus Lynx 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing