Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
ON
Only Nvidia_12684
Only · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Qwen2.5 Math 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
MI
nbeerbower mistral nemo gutenberg 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
MI
Radintloom Mistral 7b Fusion
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AI
AIdenU SOLAR 10.7b ko Y24 v0.1
Aidenu · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
DE
KSU HW SEC Deepseek SFT RL AST only
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
lotfsadek llama3.1 finetuned py v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Weyaxi MetaMath NeuralHermes 2.5 Mistral 7B Linear
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
Ganidu Phi 3 mini 4k OLScience v02
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
BL
Blexus Quble test model v1 INSTRUCT v1
Blexus · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
FL
flux1 kontext dev quants
Flux · marduk191
Params11.9B
Context—
Smallest4.0 GB
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
GE
SauerkrautLM-Gemma-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
SA
saishf Ortho SOVL 8B L3
Saishf · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
BL
Bloom 560m
Bloom · RichardErkhov
Params816M
Context1K
Q4_K_M0.6 GB
BR
BroAlanTaps GPT2 large 256 15000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
NA
Narumashi-RT-11B
Narumashi · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
AE
aetherwiing MN 12B Starcannon v1
Aetherwiing · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
MI
CorticalStack mistral 7b tak stack dpo
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TA
taicheng zephyr 7b align scan 0.0 0.7 cosine 3
Taicheng · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MA
Maestrale Chat v0.3 Beta
Maestrale · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2-0.5B-Chat_SFT_DPO
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
QW
bunnycore Qwen2.5 1.5B Matrix
Qwen · RichardErkhov
Params1.78B
Context32K
Q4_K_M1.1 GB
BL
Bloom 1b1
Bloom · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
NO
Nova 13B 50 Step
Nova · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MI
mistral_2X7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
DA
datatab Yugo55A GPT
Datatab · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB