Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
SA
Salamandra 7b
Salamandra · RichardErkhov
Params7.77B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Leo Hessianai 7b P1 Llama Errors P2 GEC
Llama · RichardErkhov
Params6.74B
Context8K
Q4_K_M4.1 GB
GGUF
HJ
Hjgugugjhuhjggg mergekit ties ysreuuq
Hjgugugjhuhjggg · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
YO
Youri 7b
Youri · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
PH
wenlianghuang Phi3 finance Matt
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
JA
Japanese Stablelm 3b 4e1t Instruct
Japanese · RichardErkhov
Params2.8B
Context4K
Q4_K_M1.7 GB
GGUF
CY
z image catalyst
Cyberrealistic · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
GGUF
LL
Llama3 70B Chinese Chat
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
NE
NeuralTranslate v0.2
Neuraltranslate · Thermostatic
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GP
p208p2002 gpt2 squad nqg hl
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
EE
EEVE Korean 10.8B Instruct v4 Sft
Eeve · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
TR
travel_microsoft_DialoGPT Small
Travel · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
PH
phi 2
Phi · andrijdavid
Params2.78B
Context2K
Q4_K_M1.8 GB
Coding
GP
gpt2-large
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
DE
Deepseek Coder 6.7b Chat
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
CodingTool use
OL
akrishnan TOFU olmo 7b ft full 25epochs lr1e 05 seed42 run1
OLMo · RichardErkhov
Params6.89B
Context4K
Q4_K_M4.2 GB
GGUF
LL
AndreyRzhaksinskiy CDS CodeLlama 7b 20241112 pretrain
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
JJhooww Mistral 7B v0.2 Base ptbr
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
VO
vox jot models
Vox · IrieDinamik
Params4.43B
Context—
Q4_K_M2.9 GB
GGUF
ZE
Zephyr 7b Beta_Bio Tutor_Dpo
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DA
DaringMaid-20B-V1.1
Daringmaid · RichardErkhov
Params20B
Context4K
Q4_K_M12.0 GB
GGUF
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
MI
mistral-7b-dolphin-sft
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SM
SmolLM 135M
SmolLM · RichardErkhov
Params135M
Context2K
Q4_K_M0.1 GB
GGUF
ST
Starcoderbase 1b
StarCoder · RichardErkhov
Params1.14B
Context8K
Q4_K_M0.8 GB
Coding
QW
Qwen3 VL 8B Thinking
Qwen · NexaAI
Params8.19B
Context—
Smallest4.6 GB
GGUF
MI
laser-dolphin-mixtral-4x7b-dpo
Mistral · RichardErkhov
Params24.2B
Context32K
Q4_K_M14.6 GB
GGUF
GE
gemma-1.1-7b-it
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
DO
DolphinStar-12.5B
Dolphinstar · RichardErkhov
Params12.5B
Context32K
Q4_K_M7.5 GB
GGUF

Other quants

Keep browsing