GGUF models by ngxson
37 GGUF models quantized by ngxson on Hugging Face. Sizes run from 1M to 673B parameters. The longest trained context is 10240K tokens.
LL
Meta Llama 3.1 8B Instruct
Llama · ngxson
Params8.03B
Context128K
Smallest8.5 GB
UL
ultravox wip ggml do not use
Ultravox · ngxson
Params683M
Context—
Smallest2.7 GB
LL
MiniThinky 1B Llama 3.2 Q8 0
Llama · ngxson
Params1.24B
Context128K
Smallest1.3 GB
LL
Llama 4 Maverick 17B 128E Instruct Q2 K
Llama · ngxson
Params401B
Context1024K
Smallest145.6 GB
GE
Gemma 2 2b
Gemma · ngxson
Params2.61B
Context8K
Smallest2.8 GB
DE
DeepSeek R1 Remixed IQ1 M
DeepSeek · ngxson
Params671B
Context160K
Smallest170.1 GB
LL
test llava will be deleted soon
LLaVA · ngxson
Params7.06B
Context4K
Smallest14.1 GB