Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
Llama 2 7b Hf Gpt 4 80k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
ME
mergekit community mergekit slerp iyanrea
Mergekit · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Speechless Mistral Hermes Code 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
finetune_starcoder2_3b_tokens_added
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
CD
CDS CL 7b Instruct Hf 20241002 Pretraining Exp2
Cds · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
MA
Marcoroni Neural Chat 7B v2
Marcoroni · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
MiniMoog Mergerix 7b v0.3
Minimoog · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
peft internal testing tiny dummy qwen2
Qwen · RichardErkhov
Params11M
Context—
Q4_K_M0.0 GB
LL
Dumi Llama3.2 3b Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
FA
ssmits Falcon2 5.5B Dutch
Falcon · RichardErkhov
Params5.47B
Context2K
Q4_K_M3.4 GB
LL
lianghsun Llama 3.2 Taiwan 3B
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
MU
mudler Asinello Minerva 3B v0.1
Mudler · RichardErkhov
Params2.89B
Context16K
Q4_K_M1.8 GB
NU
Nutanix checkpoint 1800 merged
Nutanix · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TI
tinyllama-1.1B-intermediate-step-715k-1.5T-dpo-lora-merged
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
PH
duy nq phi 3 mini 4k merge v4
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
MI
yjwon mp mistral7b sft1 b512 lr5e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
BFS Search mistral multi rel I2B2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama-2-7b-instruct
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GP
Gpt2 Large
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
LL
Llama-2-13b-chat-longlora-32k-sft
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MI
mlfoundations dev mistral alpaca sft sample 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
CH
ChatAYT Lora Assamble Marcoroni
Chatayt · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MA
Magpie Align MagpieLM 8B SFT v0.1
Magpie · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
KO
KoSOLAR-10.7B-v0.2
Kosolar · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
LL
LLama 3.1 8B HyperNova
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
EL
Eleuther Pythia70M Hh Dpo
Eleuther · RichardErkhov
Params70M
Context2K
Q4_K_M0.0 GB
LL
Llama 3.2 3b It Ecommerce ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GR
Iq Code Evmind v1 Granite 8b Instruct
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
LL
mlfoundations dev llama2 oh teknium scaling down random 0.5
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
ED
edbeeching EleutherAI pythia 2.8b
Edbeeching · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB