Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
IS
ishantd unyielding mustard agouti
Ishantd · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
ME
megatron-gpt2-345m-evol_instruct_v2
Megatron · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
simonycl llama 3.1 8b instruct ultrafeedback agg judge
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
ML
mlfoundations dev oh v1.3 opengpt x2
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
YI
Yi 6b Dpo v0.2
Yi · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
QW
Qwen3.8-27B
Qwen · Arslan2703
Params27.3B
Context256K
Smallest8.4 GB
ReasoningTool useVision
LL
Security Llama3.2 3B
Llama · prithivMLmods
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
MI
nbeerbower mistral nemo bophades 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
LL
kenhktsui llama3.2 3b instruct thinking sft merged
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
ReasoningTool use
PH
Phi 3 Mini 128k Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama-2-7b-german-assistant-v3
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
KU
Kukedlc Fasciculus Arcuatus 7B slerp
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
llama-3-8B-Instruct-function-calling
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
QW
Qwen3.6-27B-Uncensored-HauhauCS-Balanced
Qwen · CCSSNE
Params26.9B
Context256K
Smallest10.0 GB
ReasoningTool useVision
LL
LDCC Instruct Llama 2 Ko 13B v4.2.8
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
PH
third intellect Phi 3 mini 4k instruct orca math word problems 200k model 16bit
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
LL
Ko En Llama2 13b Aligned
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh v3.1 wo airoboros
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
douy parrot llama 2 13B epoch3
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
KU
Kukedlc Neural Krishna Multiverse 7b v2
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
NeuralPipe 7B Slerp
Neuralpipe · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · cvgro
Params1.86B
Context256K
Smallest0.9 GB
GGUF
XL
xLAM 7b R
Xlam · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
RH
rho-math-7b-interpreter-v0.1
Rho · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SY
SystemGemma2 2b It
Systemgemma · RichardErkhov
Params2.61B
Context32K
Q4_K_M1.7 GB
GGUF
DE
deepseek-coder-1.3b-instruct
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
LL
Llama 2 7b chat_FFT_CodeAlpaca 20k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
genloop medical insurance llama 3.1 8B instruct merged
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
GI
giraffe176 Starling Monarch Westlake Garten 7B v0.1
Giraffe · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing