Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
AL
AlphaMist7B Slr v4 Slow
Alphamist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
HO
Holarissun gpt2 sft tldr
Holarissun · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
MI
mlfoundations dev hp ablations mistral lr2e 6 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
microsoft Phi 3 medium 4k instruct
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.6 GB
GI
Giraffe-v2-70b-32k
Giraffe · RichardErkhov
Params69B
Context32K
Q4_K_M41.4 GB
DE
Srijith rkr deepseek base 1e 3 NO cot all samples 300 ckpt
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
LL
akshit Gupta llamanew1merged FinetunedByAG
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
CR
crestf411 L3.1 8B sunfall v0.6.1 dpo
Crestf · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
jan hq llama3 s instruct v0.3 checkpoint 3000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama3 S Instruct v0.3 Checkpoint 7000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
ON
Only_Mini
Only · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
PY
Pythia 1b
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
YO
yoshility model merge dare ties v1
Yoshility · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GR
GritLM 7B
Gritlm · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
lmlab-mistral-1b-untrained
Mistral · RichardErkhov
Params1.13B
Context32K
Q4_K_M0.7 GB
LL
Llama3 8B Dpo Sft v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
YU
yuntian deng gpt2 implicit cot multiplication 20 digits
Yuntian · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
MI
mlfoundations dev hp ablations mistral adambeta2 0.95 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral 7B Instruct KTO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Tess 2.0 Mixtral 8x7B
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
OP
OpenLLM Ro RoMistral 7b Instruct 2024 05 17
Openllm · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
mlfoundations dev mistral 7b 0 3 oh dcft v3.1 llama 3.1 8b
Llama · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
NU
AI MO NuminaMath 72B TIR
Numinamath · RichardErkhov
Params72.7B
Context128K
Q4_K_M47.4 GB
QW
Pretrain Qwen 500M
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
LL
Llama 2 Ko 7b
Llama · RichardErkhov
Params6.86B
Context2K
Q4_K_M4.2 GB
LL
Llama 2 13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GP
Gpt2 Context_Generator
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB