Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
HE
Heejindo rationale model e3 save5000 f2
Heejindo · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
ML
mlfoundations dev OH original wo caseus custom
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
RA
RakutenAI 7B
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
DE
deepseek-coder-6.7b-chat
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
QW
natong19 Qwen2 7B Instruct abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
JA
Jais Adapted 7b
Jais · RichardErkhov
Params7B
Context4K
Q4_K_M4.3 GB
GGUF
LL
Llama 2 7b Hf Llama2 Chat 80k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
pantelis ninja unsloth Qwen2.5 3B Instruct dtype bfloat16 r 64 lr 0.0007
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
GGUF
LL
Meta Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
CA
Cartinoe5930 KoRAE 13b
Cartinoe · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
LL
LDCC Instruct Llama 2 Ko 13B v4
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
GE
Google_Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
PH
IntelLabs sqft phi 3 mini 4k 50 base
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
chinese llama 2 13b 16k
Llama · shaowenchen
Params13.3B
Context16K
Smallest5.6 GB
GGUF
BG
CamiloGC93 bge large en v1.5 soft skills
BGE · RichardErkhov
Params334M
Context512
Q4_K_M0.2 GB
Embedding
DE
Deepseek Coder 1.3B Kexer
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
LL
Llama 3.1 Tulu 3 8B SFT
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Vision
GI
giux78 zefiro 7b sft qlora ITA v0.5
Giux · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
TE
TeeZee Bielik SOLAR LIKE 10.7B Instruct v0.1
Teezee · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
KO
KoRAE 13b
Korae · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
OC
octo planner
Octo · NexaAI
Params3.82B
Context128K
Q4_K_M2.4 GB
Coding
LL
Radiantloom Llama 70b Instruct
Llama · RichardErkhov
Params69B
Context32K
Q4_K_M41.4 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · maxidistination
Params1.86B
Context256K
Smallest0.9 GB
GGUF
MA
ManthanKulakarni TinyLlama 1.1B Text2SQL
Manthankulakarni · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
PH
ihughes15234 phi35 tictactoe dpo5epoch
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
BR
Breeze-7B-Instruct-v1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
GGUF
IN
Internlm2_5 7b Chat Abliterated
InternLM · RichardErkhov
Params7.74B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Llama 3 LewdPlay 8B Evo
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
DE
Polaris 7B Preview
DeepSeek · quantized4all
Params7.62B
Context128K
Q4_K_M4.7 GB
ReasoningTool use

Other quants

Keep browsing