Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
IN
inswave AISquare Instruct yi ko 6b v0.9.26
Inswave · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
BR
braindao iq code evmind 0.5b instruct v0.2411.3 193
Braindao · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
MI
Agnuxo Mamba Codestral 7B Instruct CODE Python Spanish English Asistant 16bit v2
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
ZE
zephyr-7b-dpo-full
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Meta-Llama-3-8B-Instruct-dequantized
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
ML
mlfoundations dev oh v1.3 evol instruct x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2.5 Coder 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
YU
yuntaeyang KoSOLAR 10.7B dpo v1.0
Yuntaeyang · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
CO
ConvAI-9b
Convai · RichardErkhov
Params8.99B
Context32K
Q4_K_M5.4 GB
PA
pankajmathur orca mini 3b
Pankajmathur · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
WI
WizardVicuna Uncensored 3B 0719
Wizardvicuna · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
BH
bhenrym14 airoboros 33b gpt4 1.4.1 PI 8192 fp16
Bhenrym · RichardErkhov
Params32.5B
Context2K
Q4_K_M19.6 GB
JW
jwkweon CUBOX SOLAR 10.7B DPO v0.1
Jwkweon · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.5 GB
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr1e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 3B Instruct JaMagpie Preview
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
LL
gpjt Qwen1.5 0.5B openassistant guanaco llama2 format
Llama · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
GE
Gemma 2b It Bnb 4bit
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
MI
disi unibo nlp mistral SFT medqa triples nographeval cot
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
BH
bhavinjawade SOLAR 10B OrcaDPO Jawade
Bhavinjawade · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
ML
mlfoundations dev oh v1.3 slim orca x.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DA
DavidAU MN Dark Planet TITAN 12B
Davidau · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
ML
mlfoundations dev oh v1.3 evol instruct x8
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
dolphin-2.9.2-qwen2-7b
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
BL
v000000 L3 8B BlueSerpentine
Blueserpentine · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
UN
uniwiz-7B-v0.1
Uniwiz · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Agnuxo Llama 3.1 Minitron 4B Instruct CODE Python Spanish English Asistant 16bit v2
Llama · RichardErkhov
Params4.54B
Context128K
Q4_K_M2.8 GB
LL
MMed Llama 3 8B EnIns
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB