Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Orenguteng Llama 3 8B LexiFun Uncensored V1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 13b
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
MI
NousResearch Nous Hermes 2 Mixtral 8x7B DPO
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
Tool use
LL
nota ai st llama 1 5.5b ppl
Llama · RichardErkhov
Params5.52B
Context2K
Q4_K_M3.3 GB
GGUF
BR
BroAlanTaps GPT2 large 128 10000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
KO
Tinyllama 2B
Kobbletinyv · Aculi
Params2.07B
Context2K
Smallest0.8 GB
GGUF
RE
realPCH kosolra wiki QA
Realpch · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
GGUF
LL
llama3-8b-math-sft-mix-2-0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
jpalgo llama 3.1 sl12k overwrite v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
Qwen2.5 Coder 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
YI
Yi 1.5 9B Chat
Yi · RichardErkhov
Params8.83B
Context8K
Q4_K_M5.3 GB
GGUF
ST
StarCoder
StarCoder · RichardErkhov
Params1.14B
Context8K
Q4_K_M0.8 GB
Coding
MA
mamba-2.8b-hf
Mamba · jpodivin
Params2.77B
Context1024K
Smallest1.4 GB
GGUF
PH
Phi 3.5 Mini IFRS Accountant
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
AK
akshit Gupta smol360M fullnew merged FinetunedByAG
Akshit · RichardErkhov
Params362M
Context2K
Q4_K_M0.3 GB
GGUF
FP
FPHam L3 8B Everything COT
Fpham · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SU
supermario v2
Supermario · andrijdavid
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GP
sp8108 gpt2 finetuned ros2 v3
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
QW
trl internal testing tiny Qwen2ForCausalLM 2.5 Coder
Qwen · RichardErkhov
Params11M
Context—
Q4_K_M0.0 GB
Coding
WE
Wenzhong-GPT2-110M
Wenzhong · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
ML
mlfoundations dev oh dcft v1.1 no curation
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
OP
OpenELM 1_1B IPO
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Meta LLama 3 Cat A LLama 70b
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
MI
cognitivecomputations dolphin 2.6 mistral 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Suzume Llama 3 8B Multilingual
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
KA
Katayoon DPO ONLY Zephyr 7B 3 iter 2
Katayoon · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
VO
vonPipe selfiesPretrainInstruct
Vonpipe · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SO
SOLAR 10.7B Instruct STOCK_SOLAR Slerp
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF

Other quants

Keep browsing