Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
GE
SauerkrautLM-Gemma-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
SA
saishf Ortho SOVL 8B L3
Saishf · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
BL
Bloom 560m
Bloom · RichardErkhov
Params816M
Context1K
Q4_K_M0.6 GB
BR
BroAlanTaps GPT2 large 256 15000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
NA
Narumashi-RT-11B
Narumashi · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
AE
aetherwiing MN 12B Starcannon v1
Aetherwiing · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
MI
CorticalStack mistral 7b tak stack dpo
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TA
taicheng zephyr 7b align scan 0.0 0.7 cosine 3
Taicheng · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MA
Maestrale Chat v0.3 Beta
Maestrale · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2-0.5B-Chat_SFT_DPO
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
JI
jina embeddings v2 base en
Jina · gpustack
Params137M
Context8K
Q4_K_M0.1 GB
QW
bunnycore Qwen2.5 1.5B Matrix
Qwen · RichardErkhov
Params1.78B
Context32K
Q4_K_M1.1 GB
BL
Bloom 1b1
Bloom · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
NO
Nova 13B 50 Step
Nova · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MI
mistral_2X7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
DA
datatab Yugo55A GPT
Datatab · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
OpenBuddy openbuddy llama3.1 8b v22.3 131k
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
CA
CatPPT base
Catppt · andrijdavid
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
incisolr gemma 2b instruct ft datascienceinterview qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
UCLA AGI Llama 3 Instruct 8B SPPO Iter3
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Llama 2 KoEn 13B
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
DE
KSU HW SEC Deepseek SFT RL DFG only
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
PlatYi 34B Llama Q v2
Llama · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
OL
OLMo 7B 0724 Hf
OLMo · aifoundry-org
Params6.89B
Context4K
Smallest1.6 GB
FU
FusionNet_7Bx2_MoE_14B
Fusionnet · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
MI
Mistral Class Tutor 7b Ep3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 2 13b Chat Hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB