Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
gradientai Llama 3 70B Instruct Gradient 262k
Llama · RichardErkhov
Params70.6B
Context256K
Q4_K_M42.5 GB
OP
OpenLLM Ro RoLlama3 8b Instruct
Openllm · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
inswave AISquare Instruct llama2 koen 13b v0.9.21
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
WE
Weyaxi OrcaMini Platypus2 13B QLoRA 0.80 epoch
Weyaxi · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
AH
Ahma 3B
Ahma · RichardErkhov
Params3.63B
Context2K
Q4_K_M2.8 GB
FR
frameai ChatFrame Instruct Persian Small
Frameai · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
SO
SOLAR 10.7B Merge Dpo
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
SU
suehyunpark potpourri 8b inst fft induction bc trajectory max10 per task check answer
Suehyunpark · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
OpenMia Indo Mistral 7b v3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
BR
Breeze 7B Base V0_1
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
BI
bikkibhagya93 gita text generation gpt2
Bikkibhagya · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
QW
Huihui Qwen3.8 27B Abliterated
Qwen · Ivanuk4PDA
Params27.3B
Context256K
Smallest10.9 GB
LL
meta llama Llama 2 13b chat hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MR
MrRobotoAI Hathor v3
Mrrobotoai · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PY
Pythia 410m Deduped v0
Pythia · RichardErkhov
Params405M
Context2K
Q4_K_M0.3 GB
JA
Japanese Stablelm Instruct Gamma 7b
Japanese · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
MI
Mistral7B PairRM SPPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
hiaoxui Llama3.1 20M
Llama · RichardErkhov
Params20M
Context—
Q4_K_M0.0 GB
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
SI
siree921 gita text generation gpt2
Siree · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
LL
FrancescoBuda Llama ICD coder 3B merged 1ep
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
MI
K2S3 Mistral 7b v1.47
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
QW
Qwen 2.5 3b Evol CoT
Qwen · RichardErkhov
Params3.4B
Context32K
Q4_K_M2.1 GB
MI
Mistral Instruct DPO Y24 v2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PY
Pythia 1b Sft Hh
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
MI
Locutusque Hyperion 2.1 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr1e 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
PH
Phi 3 Mini 4k Instruct Dpo
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB