Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
MI
synapsellm-7b-mistral-v0.4-preview2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
tinyllama-1.1b-sum-dpo-full_LR1e-7_2epochs
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
MA
Marcoroni Neural Chat 7B v2
Marcoroni · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
juhwanlee llmdo Mistral 7B case 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev hp ablations mistral epoch3 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
20240105_mistral Step50
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
YI
hongzoh Yi 6B Open Platypus v2
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
PH
myfi parser phi 3.5 mini instruct epochs 4
Phi · RichardErkhov
Params3.82B
Context8K
Q4_K_M2.3 GB
GGUF
PY
Pythia 2.8b
Pythia · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
IN
inswave AISquare Instruct yi ko 6b v0.9.26
Inswave · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
BR
braindao iq code evmind 0.5b instruct v0.2411.3 193
Braindao · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
CodingTool use
MI
Agnuxo Mamba Codestral 7B Instruct CODE Python Spanish English Asistant 16bit v2
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
Coding
ZE
zephyr-7b-dpo-full
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Meta-Llama-3-8B-Instruct-dequantized
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ML
mlfoundations dev oh v1.3 evol instruct x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 Coder 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
CodingTool use
YU
yuntaeyang KoSOLAR 10.7B dpo v1.0
Yuntaeyang · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
GGUF
CO
ConvAI-9b
Convai · RichardErkhov
Params8.99B
Context32K
Q4_K_M5.4 GB
GGUF
PA
pankajmathur orca mini 3b
Pankajmathur · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
WI
WizardVicuna Uncensored 3B 0719
Wizardvicuna · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
BH
bhenrym14 airoboros 33b gpt4 1.4.1 PI 8192 fp16
Bhenrym · RichardErkhov
Params32.5B
Context2K
Q4_K_M19.6 GB
GGUF
JW
jwkweon CUBOX SOLAR 10.7B DPO v0.1
Jwkweon · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.5 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr1e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 3B Instruct JaMagpie Preview
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
LL
gpjt Qwen1.5 0.5B openassistant guanaco llama2 format
Llama · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
GGUF
GE
Gemma 2b It Bnb 4bit
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF

Other quants

Keep browsing