Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
KU
Kukedlc SuperMente 7B v4
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
kykim0 Llama 2 7b ultrachat syn
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
RE
reflect_llm8B_om2-mstlrg-300k460k-llm33-130k-t12_SFTt1-DPOsPSDPt1_b1
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
SA
SanGuoYanYi 6b
Sanguoyanyi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
LL
KISTI KONI KONI Llama3 8B Merged 20240724
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
read1337 Llama 3.1 8B bnb 4bit marketing
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
CO
Yi Coder 9B
Coder · legraphista
Params8.83B
Context128K
Smallest2.0 GB
Coding
GR
Granite 7b Instruct
Granite · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
PatronusAI Llama 3 Patronus Lynx 70B Instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
MI
m8than mistral 7b instruct 0.2
Mistral · RichardErkhov
Params7.24B
Context—
Q4_K_M1.1 GB
GGUF
QW
Huihui Qwen3.6 35B A3B Claude 4.7 Opus Abliterated
Qwen · Gustav0-Freind
Params35.5B
Context256K
Q4_K_M21.7 GB
ReasoningTool useVision
ZE
Zephyr 7b Beta
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DY
DynaGuard AIO
Dynaguard · prithivMLmods
Params1.72B
Context40K
Q4_K_M1.1 GB
ReasoningTool use
MI
MiniCPM5 1B
MiniCPM · eaddario
Params1.08B
Context128K
Smallest0.0 GB
GGUF
LL
Llama 3 8B ArliAI Formax v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
NO
Noromaid-7B-0.4-DPO
Noromaid · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RO
RoLlama2-7b-Chat
Rollama · legraphista
Params6.74B
Context4K
Smallest1.5 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Mrt45
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
LL
Llama 3 8B Instruct 262k
Llama · Monor
Params8.03B
Context256K
Q4_K_M4.9 GB
GGUF
DE
DeepCoder 14B Preview
Deepcoder · Joumdane
Params14.8B
Context128K
Q4_K_M9.0 GB
CodingReasoningTool use
NO
nothingiisreal MN 12B Starcannon v3
Nothingiisreal · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
scb10x llama 3 typhoon v1.5x 70b instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
LL
LLaMA 3 8B SFR Iterative DPO R
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
FL
PornMaster Klein 9b
Flux · fatangel420
Params8.19B
Context256K
Q4_K_M5.0 GB
Tool use
VI
vicgalle ConfigurableHermes 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
PH
Phi 3.5 Mini Instruct
Phi · legraphista
Params3.82B
Context128K
Smallest0.8 GB
Coding
DE
DeepSeek-OCR
DeepSeek · NexaAI
Params2.93B
Context—
Smallest1.7 GB
GGUF
LL
Llama 3 Instruct 8B SimPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing