Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
OL
OLMoE 1B 7B 0924
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
GGUF
SY
Synatra_TbST02M_IN01
Synatra · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
LL
Zhao Ching TWLLM Llama2 Extend VTLoss Enhanced
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
ME
MetaMath Cybertron
Metamath · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ME
MelangeB-70b
Melangeb · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
GE
jstredacted gemma 2b instruct trivia qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
HU
ai human lab EEVE Korean Instruct 10.8B expo
Human · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
LL
Gemma 2b
Llama · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
nsmagar llama 2 7b law korean
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
OP
Openchat_3.5 Gpt 4 80k
Openchat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
LL
nbeerbower llama3.1 gutenberg 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
TU
Tulu 2 Dpo 13b
Tulu · RichardErkhov
Params13B
Context8K
Q4_K_M7.9 GB
GGUF
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
SW
v000000 SwallowMaid 8B L3 SPPO abliterated
Swallowmaid · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
CA
Calme-7B-Instruct-v0.1.1
Calme · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
failspy Llama 3 8B Instruct abliterated
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
VI
it-5.3-fp16-32k
Vikhrmodels · Vikhrmodels
Params7.63B
Context32K
Q4_K_M4.6 GB
GGUF
CA
calme-2.3-legalkit-8b
Calme · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Mistral Megamerge Dare 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AL
Alphacode AI AlphaMist7B slr v4 slow
Alphacode · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B Instruct IPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SA
SanjiWatsuki neural chat 7b v3 3 wizardmath dare me
Sanjiwatsuki · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
StopCarbon 10.7B v5
Stopcarbon · andrijdavid
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
PY
pythia1b-sft-tldr
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
DA
DAVinCI Yi Ko 6B v0.8
Davinci · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
AR
arcee ai raspberry 3B
Arcee · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
LL
RohitSuresh15 llama3.2 1b medical v1
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
SR
sravankumarmsb gita text generation gpt2
Sravankumarmsb · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
OP
openbuddy-yi1.5-9b-v21.1-32k
Openbuddy · RichardErkhov
Params8.85B
Context32K
Q4_K_M5.3 GB
GGUF
JA
Japanese Starling ChatV 7B
Japanese · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing