Nemotron GGUF models

940 GGUF models in the Nemotron family, quantized by 372 uploaders. Sizes run from 10M to 550B parameters. The longest trained context is 1048K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
NE
nemotron 3.5 asr streaming 0.6b
Nemotron · handy-computer
Params638M
Context—
Q4_K_M0.5 GB
GGUF
NE
nemotron-3.5-asr-streaming-0.6b.q8_0
Nemotron · nvidia
Params648M
Context—
Smallest0.7 GB
GGUF
NE
NVIDIA-Nemotron-3.5-Lightning-30B-A3B
Nemotron · ggml-org
Params31.6B
Context1024K
Smallest1.2 GB
GGUF
NE
NVIDIA Nemotron 3.5 Bolt 74B A9B 24GB
Nemotron · nvidia
Params74B
Context256K
Smallest22.0 GB
Vision
NE
Nemotron 3 Embed 8B BF16
Nemotron · Abiray
Params7.95B
Context256K
Q4_K_M4.9 GB
Embedding
NE
Nemotron-3-Diarization.q8_0
Nemotron · nvidia
Params99M
Context—
Smallest0.1 GB
GGUF
NE
.nemotron-speech-streaming-en-0.6b.q8_0.gguf
Nemotron · nvidia
Params628M
Context—
Smallest0.7 GB
GGUF
NE
NVIDIA Nemotron 3.5 Bolt E4B
Nemotron · unsloth
Params7.52B
Context128K
Smallest0.0 GB
GGUF
NE
Nemotron 3 Embed 1B Q4 K M
Nemotron · zenmagnets
Params1.14B
Context256K
Q4_K_M0.7 GB
Embedding
NE
Nemotron 3 Diarization
Nemotron · audio-cpp
Params99M
Context—
Smallest0.1 GB
GGUF
NE
NVIDIA Nemotron 3.5 Bolt 74B A9B
Nemotron · unsloth
Params74B
Context256K
Smallest1.0 GB
Vision
NE
NVIDIA Nemotron 3.5 Super VL 120B A12B
Nemotron · lmstudio-community
Params124B
Context1024K
Q4_K_M70.1 GB
Vision
NE
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B
Nemotron · ggml-org
Params1.66B
Context1024K
Smallest1.4 GB
ReasoningTool useVision
NE
Nemotron 3.5 Bolt E4B
Nemotron · nvidia
Params7.52B
Context64K
Q4_K_M6.2 GB
ReasoningTool use
NE
NVIDIA Nemotron 3 Nano 4B
Nemotron · lmstudio-community
Params3.97B
Context1024K
Q4_K_M2.8 GB
GGUF
NE
Nvidia-Nemotron-3.5-Lightning-30B-A3B
Nemotron · unsloth
Params32.9B
Context1024K
Q4_K_M25.3 GB
GGUF
NE
Nemotron-3-Nano-30B-A3B
Nemotron · unsloth
Params31.6B
Context1024K
Q4_K_M24.6 GB
Coding
NE
NVIDIA Nemotron 3.5 Lightning 30B A3B
Nemotron · bartowski
Params32.9B
Context1024K
Q4_K_M25.5 GB
GGUF
NE
Nemotron 3 Nano Omni 30B A3B Reasoning
Nemotron · lmstudio-community
Params31.6B
Context256K
Q4_K_M24.5 GB
ReasoningVision
NE
NVIDIA Nemotron 3.5 Super VL 120B A12B
Nemotron · unsloth
Params1.63B
Context1024K
Smallest1.3 GB
Vision
NE
NVIDIA Nemotron Nano 9B v2
Nemotron · bartowski
Params8.89B
Context1024K
Q4_K_M6.5 GB
GGUF
NE
Nemotron Speech Streaming EN
Nemotron · handy-computer
Params618M
Context—
Q4_K_M0.5 GB
GGUF
NE
NVIDIA Nemotron 3.5 Bolt 74B A9B 24GB LiLiCorr
Nemotron · nvidia
Params1.11B
Context256K
Smallest1.0 GB
GGUF
NE
Models Nvidia Nemotron Orchestrator 8B
Nemotron · MaziyarPanahi
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
NE
Models Nvidia NVIDIA Nemotron Nano 12B v2
Nemotron · MaziyarPanahi
Params12.3B
Context1024K
Q4_K_M7.5 GB
GGUF
NE
NVIDIA Nemotron 3.5 Transcribe 30B A4B
Nemotron · AtomicChat
Params32.9B
Context1024K
Smallest0.1 GB
GGUF
NE
NVIDIA-Nemotron-3.5-Bolt-74B-A9B
Nemotron · AtomicChat
Params74B
Context256K
Q4_K_M51.7 GB
GGUF
NE
NVIDIA Nemotron 3 Nano 4B
Nemotron · nvidia
Params3.97B
Context1024K
Q4_K_M2.8 GB
Tool use
NE
NVIDIA Nemotron 3.5 Lightning 30B A3B
Nemotron · AtomicChat
Params32.9B
Context1024K
Q4_K_M25.0 GB
ReasoningTool use
NE
NVIDIA Nemotron Labs Dory1.0 4B
Nemotron · nvidia
Params3.98B
Context256K
Smallest3.7 GB
ReasoningTool use

About the Nemotron GGUF models

Nemotron is represented on ggufmodels.com by 940 GGUF models, quantized by 372 uploaders on Hugging Face. The most downloaded are nemotron 3.5 asr streaming 0.6b, nemotron-3.5-asr-streaming-0.6b.q8_0 and NVIDIA-Nemotron-3.5-Lightning-30B-A3B. Parameter counts run from 10M to 550B, and the longest trained context window is 1048K tokens.

A Q4_K_M file of a Nemotron model is between 0.2 to 379.6 GB, which is the quant most people start with. Most files come from mradermacher (207), bartowski (35) and DevQuasar (34). The quants you will meet most often are Q4_K_M (587), Q8_0 (459), Q5_K_M (431) and Q6_K (423), where the number is how many models offer that quant.

Frequently asked questions

Which Nemotron GGUF should I download?

The most downloaded Nemotron GGUF here is nemotron 3.5 asr streaming 0.6b. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Nemotron model need?

Q4_K_M files of Nemotron models on this site range from 0.2 to 379.6 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Nemotron GGUF files come from?

The most common quantizers here are mradermacher, bartowski and DevQuasar. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing