GGUF models by NonMiFrega
38 GGUF models quantized by NonMiFrega on Hugging Face. Sizes run from 438M to 12.9B parameters. The longest trained context is 1024K tokens.
MI
Mistral 7B Instruct v0.2
Mistral · NonMiFrega
Params7.24B
Context32K
Q4_K_M4.4 GB
SM
SmolLM2 1.7B Instruct
SmolLM · NonMiFrega
Params1.71B
Context8K
Q4_K_M1.1 GB
MI
mistral instruct frankenmerge Q4 K M
Mistral · NonMiFrega
Params8.99B
Context32K
Q4_K_M5.4 GB
LL
Llama 3.2 1B Instruct
Llama · NonMiFrega
Params1.24B
Context128K
Q4_K_M0.8 GB
GE
Gemma 2 9b
Gemma · NonMiFrega
Params9.24B
Context8K
Q4_K_M5.8 GB
GE
Gemma 2 2b
Gemma · NonMiFrega
Params2.61B
Context8K
Q4_K_M1.7 GB
MI
mistral instruct slerp Q4 K M
Mistral · NonMiFrega
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.1 8B
Llama · NonMiFrega
Params8.03B
Context128K
Q4_K_M4.9 GB