GGUF models by NonMiFrega

38 GGUF models quantized by NonMiFrega on Hugging Face. Sizes run from 438M to 12.9B parameters. The longest trained context is 1024K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Mistral 7B Instruct v0.2
Mistral · NonMiFrega
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SM
SmolLM2 1.7B Instruct
SmolLM · NonMiFrega
Params1.71B
Context8K
Q4_K_M1.1 GB
GGUF
MI
mistral instruct frankenmerge Q4 K M
Mistral · NonMiFrega
Params8.99B
Context32K
Q4_K_M5.4 GB
GGUF
LL
Llama 3.2 1B Instruct
Llama · NonMiFrega
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
GE
Gemma 2 9b
Gemma · NonMiFrega
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
GE
Gemma 2 2b
Gemma · NonMiFrega
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
MI
mistral instruct slerp Q4 K M
Mistral · NonMiFrega
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.1 8B
Llama · NonMiFrega
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF

Keep browsing