Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
Openbuddy Mistral7B v19.1 4k
Mistral · RichardErkhov
Params7.28B
Context32K
Q4_K_M4.4 GB
MI
Amod mental health therapy mistral 7b ins SFT
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
Stablecode Instruct Alpha 3b
Stablecode · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
AN
Anima P3 Turbo AIO Q4 K
Anima · n-Arno
Params2.81B
Context—
Smallest1.8 GB
LL
JS CodeLlama 7b 20241203
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QG
qgallouedec tiny GPTNeoXForCausalLM
Qgallouedec · RichardErkhov
Params—
Context—
Q4_K_M0.0 GB
MI
Mistral 7B Instruct RDPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
NE
NESPED GEN TinyLlama1B selector and SQL v1 all 16500steps
Nesped · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
SA
safmn sr
Safmn · cstr
Params2M
Context—
Smallest0.0 GB
SO
Solar Pro Preview Instruct
Solar · RichardErkhov
Params22.1B
Context4K
Q4_K_M13.3 GB
CH
ChatWaifu_12B_v2.0
Chatwaifu · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
TI
TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
ML
mlfoundations dev oh dcft v3 sharegpt format sedrick
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Meta Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
QW
Qwen3.8 Flash Next GSQ RCO MoREL PB3 208
Qwen · oisee
Params105B
Context256K
Smallest3.0 GB
KU
Kukedlc Neural 4 ARC 7b
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
YeungNLP firefly qwen1.5 en 7b unsloth
Qwen · RichardErkhov
Params7.72B
Context32K
Q4_K_M1.0 GB
EL
Elizezen Berghof ERP 7B
Elizezen · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GE
gemma-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
UN
UNfilteredAI-1B
Unfilteredai · RichardErkhov
Params1.06B
Context2K
Q4_K_M0.6 GB
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
AndreyRzhaksinskiy CDS CodeLlama Ins 7b E2E 20241022 experiment
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
Gemma 2 9B
Llama · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
ML
mlabonne BigQwen2.5 Echo 47B Instruct
Mlabonne · RichardErkhov
Params47.4B
Context32K
Q4_K_M28.6 GB
LL
rubra ai Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.9B
Context8K
Q4_K_M5.4 GB
LL
Regulus Qwen3 R1 Llama Distill
Llama · prithivMLmods
Params1.72B
Context40K
Q4_K_M1.1 GB
EX
explorewithai ChatFrame Uncensored Instruct Small
Explorewithai · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
IN
interview eval zephyr 7b stem train
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SH
shadowml DareBeagel 2x7B
Shadowml · RichardErkhov
Params14.4B
Context32K
Q4_K_M7.8 GB