Nemotron GGUF models
940 GGUF models in the Nemotron family, quantized by 372 uploaders. Sizes run from 10M to 550B parameters. The longest trained context is 1048K tokens.
About the Nemotron GGUF models
Nemotron is represented on ggufmodels.com by 940 GGUF models, quantized by 372 uploaders on Hugging Face. The most downloaded are nemotron 3.5 asr streaming 0.6b, nemotron-3.5-asr-streaming-0.6b.q8_0 and NVIDIA-Nemotron-3.5-Lightning-30B-A3B. Parameter counts run from 10M to 550B, and the longest trained context window is 1048K tokens.
A Q4_K_M file of a Nemotron model is between 0.2 to 379.6 GB, which is the quant most people start with. Most files come from mradermacher (207), bartowski (35) and DevQuasar (34). The quants you will meet most often are Q4_K_M (587), Q8_0 (459), Q5_K_M (431) and Q6_K (423), where the number is how many models offer that quant.
Frequently asked questions
Which Nemotron GGUF should I download?
The most downloaded Nemotron GGUF here is nemotron 3.5 asr streaming 0.6b. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.
How much VRAM does a Nemotron model need?
Q4_K_M files of Nemotron models on this site range from 0.2 to 379.6 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.
Where do Nemotron GGUF files come from?
The most common quantizers here are mradermacher, bartowski and DevQuasar. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.