GGUF models by kromquant
78 GGUF models quantized by kromquant on Hugging Face. Sizes run from 8.03B to 12B parameters. The longest trained context is 4192K tokens.
MI
Milkyway
Milkyway · kromquant
Params12B
Context128K
Smallest12.7 GB
OD
Odious
Odious · kromquant
Params12B
Context128K
Smallest12.7 GB
TE
testing tan 001 GGUFs
Testing · kromquant
Params8.03B
Context128K
Smallest8.5 GB
TE
testing eve 001 GGUFs
Testing · kromquant
Params8.03B
Context128K
Smallest8.5 GB
TE
testing vim 004 GGUFs
Testing · kromquant
Params8.03B
Context128K
Smallest8.5 GB
TE
testing tact 001 GGUFs
Testing · kromquant
Params8.03B
Context128K
Smallest8.5 GB
TE
testing hip 001 GGUFs
Testing · kromquant
Params8.03B
Context128K
Smallest8.5 GB
MA
Mantle
Mantle · kromquant
Params12B
Context128K
Smallest12.7 GB
SU
Subfusc
Subfusc · kromquant
Params12B
Context128K
Smallest12.7 GB
TE
testing page 001 GGUFs
Testing · kromquant
Params8.03B
Context1048K
Smallest8.5 GB
TE
testing wet 006 GGUFs
Testing · kromquant
Params8.03B
Context128K
Smallest8.5 GB
TE
testing lot 002 GGUFs
Testing · kromquant
Params8.03B
Context128K
Smallest8.5 GB
TW
Twilight
Twilight · kromquant
Params12B
Context128K
Smallest12.7 GB
SP
Spectre
Spectre · kromquant
Params12B
Context128K
Smallest12.7 GB
BE
Beasts
Beasts · kromquant
Params12B
Context128K
Smallest12.7 GB
TE
testing bronze 001 GGUFs
Testing · kromquant
Params8.03B
Context128K
Smallest8.5 GB
TE
testing chill 003 GGUFs
Testing · kromquant
Params8.03B
Context128K
Smallest8.5 GB
SA
Sallow
Sallow · kromquant
Params12B
Context128K
Smallest12.7 GB