Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
AL
AlphaMist7B Slr v4 Slow
Alphamist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
HO
Holarissun gpt2 sft tldr
Holarissun · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
mlfoundations dev hp ablations mistral lr2e 6 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
microsoft Phi 3 medium 4k instruct
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.6 GB
GGUF
GI
Giraffe-v2-70b-32k
Giraffe · RichardErkhov
Params69B
Context32K
Q4_K_M41.4 GB
GGUF
DE
Srijith rkr deepseek base 1e 3 NO cot all samples 300 ckpt
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
LL
akshit Gupta llamanew1merged FinetunedByAG
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
CR
crestf411 L3.1 8B sunfall v0.6.1 dpo
Crestf · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
jan hq llama3 s instruct v0.3 checkpoint 3000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama3 S Instruct v0.3 Checkpoint 7000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
ON
Only_Mini
Only · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
PY
Pythia 1b
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
YO
yoshility model merge dare ties v1
Yoshility · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GR
GritLM 7B
Gritlm · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
lmlab-mistral-1b-untrained
Mistral · RichardErkhov
Params1.13B
Context32K
Q4_K_M0.7 GB
GGUF
LL
Llama3 8B Dpo Sft v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
YU
yuntian deng gpt2 implicit cot multiplication 20 digits
Yuntian · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
MI
mlfoundations dev hp ablations mistral adambeta2 0.95 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B Instruct KTO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Tess 2.0 Mixtral 8x7B
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
OP
OpenLLM Ro RoMistral 7b Instruct 2024 05 17
Openllm · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
mlfoundations dev mistral 7b 0 3 oh dcft v3.1 llama 3.1 8b
Llama · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
NU
AI MO NuminaMath 72B TIR
Numinamath · RichardErkhov
Params72.7B
Context128K
Q4_K_M47.4 GB
GGUF
QW
Pretrain Qwen 500M
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
GGUF
LL
Llama 2 Ko 7b
Llama · RichardErkhov
Params6.86B
Context2K
Q4_K_M4.2 GB
GGUF
LL
Llama 2 13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
GP
Gpt2 Context_Generator
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF

Other quants

Keep browsing