Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
KY
kyujinpy Sakura SOLRCA Instruct DPO
Kyujinpy · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
disi unibo nlp llama3.2 1B SFT medqa triples cot
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
MI
IntelLabs sqft sparsepeft mistral 7b v0.3 50 gsm8k heu
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
SO
SOLAR 10.7B Orca Alpaca Gpt4 Lora 653
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
FI
FILM 7B
Film · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SP
Speechless Hermes Coig Lite 13b
Speechless · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
Tool use
LL
zomd AISquare Instruct llama2 koen 13b v0.9.17
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
XU
XueyingJia pythia 1b deduped hh online dpo merge
Xueyingjia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · odexus
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
TI
TinySolar-248m-4k-code-instruct
Tinysolar · RichardErkhov
Params248M
Context32K
Q4_K_M0.2 GB
Coding
QW
Qwen2-0.5B-OnlineDPO-PairRM
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
QW
Qwen2.5 72B Instruct
Qwen · RichardErkhov
Params72.7B
Context32K
Q4_K_M47.4 GB
Tool use
MI
Mistral-7b-instruct-v0.2-summ-sft-e2m
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
MI
mlfoundations dev mistral alpaca sft sample 5
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Huihui Qwythos 9B Claude Mythos 5 1M Abliterated MTP
Qwythos · eadx
Params9.2B
Context1024K
Smallest5.8 GB
ReasoningTool useVision
MI
sania963 Mistral instruct sql v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
Tool use
WE
Weyaxi openchat 3.5 1210 Seraph Slerp
Weyaxi · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IN
InternLM2
InternLM · RichardErkhov
Params19.9B
Context8K
Q4_K_M12.0 GB
GGUF
MI
InnerI InnerILLM OpenPipe Nous Yarn Mistral optimized 1228 7B slerp
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BR
BroAlanTaps GPT2 large 128 46000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
MI
Moriacrafter Mistral 7B v0.2 none DepressionDetection Reddit
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
JI
JiuZhang3.0 Synthesis 7B
Jiuzhang · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
MI
EleutherAI Mistral 7B v0.1 population first ft
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Yarn-Mistral-7b-128k-DPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Qweassdzxc
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
GE
Google Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
FA
gguf falcon mamba 7b instruct
Falcon · Felladrin
Params7.27B
Context1024K
Smallest2.8 GB
GGUF
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
RE
reflect_llm8B_Om2SftT1-Om2IpsdpG8kIpsdpIter1T1_b0.1
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use

Other quants

Keep browsing