Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
GR
Grafted Hermetic Platypus D 2x7B
Grafted · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
ML
mlfoundations dev oh teknium scaling down random 0.3
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Mistral 7B Merge 14 v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Embedding
TW
Twice-KoSOLAR-16.1B-instruct-test
Twice · RichardErkhov
Params16.1B
Context4K
Q4_K_M9.7 GB
GGUF
ST
Stopwolf DistilabelCerberus 7B slerp
Stopwolf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
Phi 3 mini mango 1
Phi · rhysjones
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
Llama3 Aloe 8B Alpha
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
IN
interview eval zephyr 7b math case 4
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
flozi00 Llama 2 13b german assistant v5
Llama · RichardErkhov
Params13.1B
Context2K
Q4_K_M7.9 GB
GGUF
ML
mlfoundations dev oh v1 2 only alpaca
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
gradientai Llama 3 70B Instruct Gradient 262k
Llama · RichardErkhov
Params70.6B
Context256K
Q4_K_M42.5 GB
GGUF
OP
OpenLLM Ro RoLlama3 8b Instruct
Openllm · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
inswave AISquare Instruct llama2 koen 13b v0.9.21
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
WE
Weyaxi OrcaMini Platypus2 13B QLoRA 0.80 epoch
Weyaxi · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
AH
Ahma 3B
Ahma · RichardErkhov
Params3.63B
Context2K
Q4_K_M2.8 GB
GGUF
FR
frameai ChatFrame Instruct Persian Small
Frameai · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
SO
SOLAR 10.7B Merge Dpo
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
SU
suehyunpark potpourri 8b inst fft induction bc trajectory max10 per task check answer
Suehyunpark · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
OpenMia Indo Mistral 7b v3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BR
Breeze 7B Base V0_1
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
GGUF
BI
bikkibhagya93 gita text generation gpt2
Bikkibhagya · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
QW
Huihui Qwen3.8 27B Abliterated
Qwen · Ivanuk4PDA
Params27.3B
Context256K
Smallest10.9 GB
ReasoningTool useVision
LL
meta llama Llama 2 13b chat hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MR
MrRobotoAI Hathor v3
Mrrobotoai · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PY
Pythia 410m Deduped v0
Pythia · RichardErkhov
Params405M
Context2K
Q4_K_M0.3 GB
GGUF
JA
Japanese Stablelm Instruct Gamma 7b
Japanese · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
MI
Mistral7B PairRM SPPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
hiaoxui Llama3.1 20M
Llama · RichardErkhov
Params20M
Context—
Q4_K_M0.0 GB
GGUF
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
SI
siree921 gita text generation gpt2
Siree · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF

Other quants

Keep browsing