Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
ST
Stablecode Instruct Alpha 3b
Stablecode · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
GGUF
LL
JS CodeLlama 7b 20241203
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QG
qgallouedec tiny GPTNeoXForCausalLM
Qgallouedec · RichardErkhov
Params—
Context—
Q4_K_M0.0 GB
GGUF
MI
Mistral 7B Instruct RDPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
NESPED GEN TinyLlama1B selector and SQL v1 all 16500steps
Nesped · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
SO
Solar Pro Preview Instruct
Solar · RichardErkhov
Params22.1B
Context4K
Q4_K_M13.3 GB
GGUF
CH
ChatWaifu_12B_v2.0
Chatwaifu · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
TI
TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
ML
mlfoundations dev oh dcft v3 sharegpt format sedrick
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Meta Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
KU
Kukedlc Neural 4 ARC 7b
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
YeungNLP firefly qwen1.5 en 7b unsloth
Qwen · RichardErkhov
Params7.72B
Context32K
Q4_K_M1.0 GB
GGUF
EL
Elizezen Berghof ERP 7B
Elizezen · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
GE
gemma-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
UN
UNfilteredAI-1B
Unfilteredai · RichardErkhov
Params1.06B
Context2K
Q4_K_M0.6 GB
GGUF
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
AndreyRzhaksinskiy CDS CodeLlama Ins 7b E2E 20241022 experiment
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Gemma 2 9B
Llama · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
QW
Ostrich 27B Qwen3.5 260411
Qwen · etemiz
Params26.9B
Context256K
Smallest7.7 GB
ReasoningTool use
ML
mlabonne BigQwen2.5 Echo 47B Instruct
Mlabonne · RichardErkhov
Params47.4B
Context32K
Q4_K_M28.6 GB
Tool use
LL
rubra ai Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.9B
Context8K
Q4_K_M5.4 GB
GGUF
EX
explorewithai ChatFrame Uncensored Instruct Small
Explorewithai · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
IN
interview eval zephyr 7b stem train
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SH
shadowml DareBeagel 2x7B
Shadowml · RichardErkhov
Params14.4B
Context32K
Q4_K_M7.8 GB
GGUF
LL
arnavgrg llama 2 7b chat nf4 fp16 upscaled
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
alpindale gemma 2b it
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
QW
Firefly Qwen1.5 En 7b Dpo
Qwen · RichardErkhov
Params7.72B
Context32K
Q4_K_M4.8 GB
GGUF
SE
SeaLLMs v3 7B
Seallms · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF

Other quants

Keep browsing