GGUF models by nold
43 GGUF models quantized by nold on Hugging Face. Sizes run from 1.35B to 46.7B parameters. The longest trained context is 512K tokens.
About nold on Hugging Face
nold has published 43 GGUF models that ggufmodels.com tracks, across families such as Mistral, StarCoder and Gemma. The most downloaded are Phi 3 mini 4k instruct function calling, CroissantLLMChat v0.1 and Mistral 7B Instruct v0.2 Neural Story.
The quants nold publishes most often are Q8_0 (43), Q5_K_M (42), Q4_K_M (42) and Q6_K (36). Their Q4_K_M files range from 0.9 to 28.5 GB. Parameter counts run from 1.35B to 46.7B, and the longest trained context window is 512K tokens.
Frequently asked questions
What GGUF models has nold published?
43 models are listed here, mainly from the Mistral, StarCoder and Gemma families. The most downloaded is Phi 3 mini 4k instruct function calling.
Which quants does nold publish?
The most common are Q8_0 (43 models), Q5_K_M (42 models), Q4_K_M (42 models) and Q6_K (36 models).