Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
ON
Only Nvidia_12684
Only · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen2.5 Math 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
Tool use
MI
nbeerbower mistral nemo gutenberg 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
MI
Radintloom Mistral 7b Fusion
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AI
AIdenU SOLAR 10.7b ko Y24 v0.1
Aidenu · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
DE
KSU HW SEC Deepseek SFT RL AST only
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
LL
lotfsadek llama3.1 finetuned py v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Weyaxi MetaMath NeuralHermes 2.5 Mistral 7B Linear
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
PH
Ganidu Phi 3 mini 4k OLScience v02
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
BL
Blexus Quble test model v1 INSTRUCT v1
Blexus · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
FL
flux1 kontext dev quants
Flux · marduk191
Params11.9B
Context—
Smallest4.0 GB
GGUF
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
GE
SauerkrautLM-Gemma-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
SA
saishf Ortho SOVL 8B L3
Saishf · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
BL
Bloom 560m
Bloom · RichardErkhov
Params816M
Context1K
Q4_K_M0.6 GB
GGUF
BR
BroAlanTaps GPT2 large 256 15000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
NA
Narumashi-RT-11B
Narumashi · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
AE
aetherwiing MN 12B Starcannon v1
Aetherwiing · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
MI
CorticalStack mistral 7b tak stack dpo
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TA
taicheng zephyr 7b align scan 0.0 0.7 cosine 3
Taicheng · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MA
Maestrale Chat v0.3 Beta
Maestrale · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2-0.5B-Chat_SFT_DPO
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
QW
bunnycore Qwen2.5 1.5B Matrix
Qwen · RichardErkhov
Params1.78B
Context32K
Q4_K_M1.1 GB
Tool use
BL
Bloom 1b1
Bloom · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
GGUF
NO
Nova 13B 50 Step
Nova · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
mistral_2X7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
DA
datatab Yugo55A GPT
Datatab · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing