Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
RE
reverie-7b
Reverie · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
ministral Ministral 3b instruct
Mistral · RichardErkhov
Params3.32B
Context128K
Q4_K_M2.0 GB
GGUF
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh dcft v1 wo camel biology gpt 4o mini
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
CodeLlama 34B Python Fp16
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
Coding
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
ML
mlfoundations dev oh teknium scaling down random 0.1
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama Guard 3 1B
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
GGUF
DB
Dbrx Instruct
Dbrx · RichardErkhov
Params208B
Context32K
Q4_K_M79.9 GB
GGUF
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
TE
Teuken 7B Base v0.6
Teuken · shahzeb171
Params8.48B
Context4K
Smallest3.8 GB
GGUF
SA
SambaLingo Russian Chat
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
GGUF
GP
gpt2-medium-italian-embeddings
Gpt · RichardErkhov
Params365M
Context1K
Q4_K_M0.2 GB
Embedding
MI
Oh Mistral bs4096_lr5_00E 06_schedulerconstant_warmup1_00E 01_minlr
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
CH
Chatgpt2
Chatgpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
GR
Granite Guardian 3.0 8b
Granite · RichardErkhov
Params8.17B
Context8K
Q4_K_M4.9 GB
GGUF
ST
Starcoder2 15b
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
Coding
TI
Tinyllama_Alpaca_Sft_Sample
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
BI
bitnet_b1_58-large
Bitnet · RichardErkhov
Params729M
Context2K
Q4_K_M0.5 GB
GGUF
SM
SmolLM 135M Instruct
SmolLM · RichardErkhov
Params135M
Context2K
Q4_K_M0.1 GB
GGUF
LL
Llama 2 7b chat_FFT_Alpaca Gpt4 Zh
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
PH
dbands Phi 3.5 mini instruct pyomo pysim coder
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
Coding
LL
Llama 3.2 3B Instruct SFT D_chosen Pref Mix
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
GE
vaniebermudez gemma 2b instruct ft squad qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
HE
Heejindo rationale model e10
Heejindo · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh dcft v1 wo camel chemistry gpt 4o mini
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MA
MathCoder L 7B
Mathcoder · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
Coding
LL
CodeLlama 7b Hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MA
MarcusCosta Sumarizadors
Marcuscosta · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF

Other quants

Keep browsing