Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr5e 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
mirlab AkaLlama llama3 70b v0.1
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
YU
Yuyuan GPT2 110M SciFi Chinese
Yuyuan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
RE
reflect_llm8B_Om2SftT1-Om2G8kOm2Ag40kIpsdpIter1T1_b0.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
RakshitAralimatti Qwen2.5 Coder 1.5B Instruct RASA CALM
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
LL
mlfoundations dev llama3 1 8b mlfoundations dev stackexchange quantumcomputing
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LD
LDCC SOLAR 10.7B
Ldcc · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
GGUF
BL
Bloomz 3b Sft Chat
Bloomz · RichardErkhov
Params3.64B
Context3K
Q4_K_M2.4 GB
GGUF
LL
Llama 3 Patronus Lynx 8B Instruct v1.1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QL
Qllama-.5B-Base-Wiki-Chat-RAG
Qllama · RichardErkhov
Params464M
Context4K
Q4_K_M0.3 GB
GGUF
MI
dolphin-2.6-mistral-7b-dpo-orca-v2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
Deepseek Coder 6.7b Base 20241020
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
TE
tenyx TenyxChat 7B v1
Tenyx · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MI
HanningZhang Mistral MetaMath 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
SA
saishf Fett Eris Mix 7B
Saishf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
Phi 3 Mini 128k Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Speechless Llama2 Luban Orca Platypus 13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
QW
abdulmannan 01 qwen 2.5 0.5b finetuned for sql generation combined dataset
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
PY
Pythia 1b Sft Hh
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Llama3 8b Ko Sft Dpo v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen1.5-MoE-A2.7B
Qwen · gdax
Params14.3B
Context8K
Q4_K_M9.5 GB
GGUF
PH
Phi 2 Insurance_Qa Sft Lora
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
GGUF
GE
Gemma 3b 002
Gemma · RichardErkhov
Params3.17B
Context8K
Q4_K_M2.0 GB
GGUF
KA
Katayoon VPO SELM Zephyr 7B iter 2
Katayoon · RichardErkhov
Params7.24B
Context32K
Q4_K_M1.5 GB
GGUF
MI
Infinity Instruct 3M 0625 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama-3-8B-Instruct-v0.4
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
johnpaulbin llama3.1 8b tokipona toki translate 1 fp16
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MA
MagicPrompt-Dalle
Magicprompt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
PH
jasonhwan phi3 redteamer
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF

Other quants

Keep browsing