GGUF models by Cran-May
92 GGUF models quantized by Cran-May on Hugging Face. Sizes run from 66M to 72.7B parameters. The longest trained context is 1000K tokens.
QW
InternLM3 8B Instruct Converted Qwen2
Qwen · Cran-May
Params8.8B
Context32K
Smallest6.1 GB
LL
Openbuddy Llama3.2 3b v23.2 131k
Llama · Cran-May
Params3.61B
Context128K
Smallest2.6 GB
MI
SLIDE v3 Preview3 Q4 K M
Mistral · Cran-May
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
SLIDE v3 Preview5 Q4 K S
Mistral · Cran-May
Params7.24B
Context32K
Smallest4.1 GB
LL
openbuddy llama3.1 8b v22.2 131k Q4 K M
Llama · Cran-May
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
Gemma-2-9B-Chinese-Chat
Gemma · Cran-May
Params9.24B
Context8K
Q4_K_M5.8 GB
GH
Ghost 8b Beta 1608
Ghost · Cran-May
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
openbuddy-mistral-22b-v21.1-32k
Mistral · Cran-May
Params22.4B
Context64K
Smallest12.7 GB
MI
openbuddy mistral2 7b v20.2 32k Q4 K M
Mistral · Cran-May
Params7.28B
Context32K
Q4_K_M4.4 GB
TE
T.E 8.1 Q4 K M
Cran-May
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
Mistral 22B v0.2 Q2 K
Mistral · Cran-May
Params22.2B
Context64K
Smallest8.3 GB
OP
Openbuddy Openllama 7b v12
Openbuddy · Cran-May
Params6.78B
Context4K
Smallest4.8 GB
QW
PARM 2 TINY 1.7B QwQ o1 Q5 K M
Qwen · Cran-May
Params1.71B
Context8K
Smallest1.2 GB
TE
Motacilla Cinerea 0308
Tempmotacilla · Cran-May
Params14.8B
Context128K
Smallest6.7 GB
LL
openbuddy llama3.1 8b v22.3 131k Q4 K M
Llama · Cran-May
Params8.03B
Context128K
Q4_K_M4.9 GB
BL
blossom-v3_1-baichuan2-13b
Blossom · Cran-May
Params13.9B
Context4K
Q4_K_M9.0 GB
LA
Lamarck 14B v0.7 Q4 K M
Lamarck · Cran-May
Params14.8B
Context128K
Q4_K_M9.0 GB
LL
Llama 3.2 3B
Llama · Cran-May
Params3.21B
Context128K
Smallest2.6 GB
QW
firefly-qwen1.5-en-14b-alpha
Qwen · Cran-May
Params14.2B
Context32K
Q4_K_M9.2 GB
QW
Qwen2 0.5B
Qwen · Cran-May
Params494M
Context128K
Smallest0.4 GB
MI
SLIDE v3 Preview6 Q4 K M
Mistral · Cran-May
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
Llama 3.1 Tulu 3.1 8B Q4 K M
Llama · Cran-May
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Mistral 22B v0.2 Q5 K M
Mistral · Cran-May
Params22.2B
Context64K
Smallest15.7 GB
MI
SLIDE v3 Preview0 Q4 K M
Mistral · Cran-May
Params7.24B
Context32K
Q4_K_M4.4 GB
UW
UwU 7B Instruct Abliterated
Uwu · Cran-May
Params7.62B
Context128K
Smallest5.4 GB
DE
DeepSeek R1 Distill Llama 8B
DeepSeek · Cran-May
Params8.03B
Context128K
Smallest4.7 GB
AW
AwA_Motacilla Cinerea 0122
Awa · Cran-May
Params8.8B
Context32K
Q4_K_M5.4 GB
GE
Gemma2 9B IT Simpo Infinity Preference
Gemma · Cran-May
Params9.24B
Context8K
Smallest6.6 GB
MI
openbuddy mistral 7b v13.1 Q5 K M
Mistral · Cran-May
Params7.28B
Context32K
Smallest5.2 GB
IN
InternLM2
InternLM · Cran-May
Params7.74B
Context32K
Smallest4.2 GB