GGUF models by CHE-72
86 GGUF models quantized by CHE-72 on Hugging Face. Sizes run from 494M to 14B parameters. The longest trained context is 128K tokens.
YI
Yi-1.5-6B-Chat
Yi · CHE-72
Params6.06B
Context4K
Smallest3.5 GB
PH
Phi 3 mini 128k instruct Q8 0
Phi · CHE-72
Params3.82B
Context128K
Smallest4.1 GB
QW
Qwen1.5-4B-Chat
Qwen · CHE-72
Params3.95B
Context32K
Q4_K_M2.5 GB
QW
Qwen1.5-4B-Chat
Qwen · CHE-72
Params3.95B
Context32K
Smallest1.9 GB
QW
Qwen2-0.5B-Instruct
Qwen · CHE-72
Params494M
Context32K
Smallest0.5 GB
QW
Qwen2-7B-Instruct
Qwen · CHE-72
Params7.62B
Context32K
Smallest4.4 GB
QW
Qwen1.5-4B-Chat
Qwen · CHE-72
Params3.95B
Context32K
Smallest2.3 GB
QW
Qwen1.5-4B-Chat
Qwen · CHE-72
Params3.95B
Context32K
Smallest2.8 GB
QW
Qwen2-7B-Instruct
Qwen · CHE-72
Params7.62B
Context32K
Smallest4.1 GB
BA
Baichuan2-7B-Chat
Baichuan · CHE-72
Params7.51B
Context4K
Smallest3.0 GB
PH
Phi 3 medium 128k instruct Q5 K M
Phi · CHE-72
Params14B
Context128K
Smallest10.1 GB
YI
Yi-1.5-6B-Chat
Yi · CHE-72
Params6.06B
Context4K
Smallest4.2 GB
QW
Qwen2-0.5B-Instruct
Qwen · CHE-72
Params494M
Context32K
Smallest0.5 GB
QW
Qwen1.5-4B-Chat
Qwen · CHE-72
Params3.95B
Context32K
Smallest2.2 GB
QW
Qwen2-7B-Instruct
Qwen · CHE-72
Params7.62B
Context32K
Smallest5.3 GB
QW
Qwen2-7B-Instruct
Qwen · CHE-72
Params7.62B
Context32K
Smallest3.8 GB
YI
Yi-1.5-6B-Chat
Yi · CHE-72
Params6.06B
Context4K
Smallest2.7 GB
QW
Qwen2-7B-Instruct
Qwen · CHE-72
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Qwen2-7B-Instruct
Qwen · CHE-72
Params7.62B
Context32K
Smallest5.4 GB
BR
Breeze-7B-Instruct-v1_0
Breeze · CHE-72
Params7.49B
Context32K
Smallest6.1 GB
YI
Yi-1.5-6B-Chat
Yi · CHE-72
Params6.06B
Context4K
Smallest4.3 GB
BR
Breeze-7B-Instruct-v1_0
Breeze · CHE-72
Params7.49B
Context32K
Smallest3.7 GB
TA
TAIDE-LX-7B-Chat
Taide · CHE-72
Params6.94B
Context4K
Smallest4.8 GB
QW
Qwen1.5-4B-Chat
Qwen · CHE-72
Params3.95B
Context32K
Smallest4.2 GB
BA
Baichuan2-7B-Chat
Baichuan · CHE-72
Params7.51B
Context4K
Smallest6.2 GB
PH
Phi 3 medium 128k instruct Q8 0
Phi · CHE-72
Params14B
Context128K
Smallest14.8 GB
BA
Baichuan2-7B-Chat
Baichuan · CHE-72
Params7.51B
Context4K
Smallest4.4 GB
BA
Baichuan2-7B-Chat
Baichuan · CHE-72
Params7.51B
Context4K
Smallest4.4 GB
YI
Yi-1.5-6B-Chat
Yi · CHE-72
Params6.06B
Context4K
Smallest3.2 GB
QW
Qwen2-7B-Instruct
Qwen · CHE-72
Params7.62B
Context32K
Smallest3.5 GB