NVIDIA Nemotron 3.5 Super VL 120B A12B needs about 82.5 GB of memory at Q4_K_M (a 70.1 GB file) with an 8K context window, which is more than a 48 GB GPU holds. This repository has 2 quants, from 70.1 GB (Q4_K_M) to 75.2 GB. Longer context needs more: at 32K tokens the KV cache alone adds about 47.8 GB.
NVIDIA Nemotron 3.5 Super VL 120B A12B – a large vision‑enabled LLM
The NVIDIA Nemotron 3.5 Super VL 120B A12B is a 120 B parameter model with 12 B active experts. It supports vision tasks and was quantized by the LM Studio community. The model is available in two GGUF variants.
Good at
- Vision‑enabled language tasks
- Large context handling
- Mixture‑of‑experts inference
Limits
- No license information provided
- Requires high VRAM for full precision
- No explicit vision model size limits
Best for
- Image captioning with long context
- Multimodal document analysis
- Large‑scale visual question answering
Who it’s for. Developers and researchers needing a high‑capacity vision‑enabled LLM. Suitable for applications that can allocate sufficient GPU memory and handle a mixture‑of‑experts architecture.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| Q4_K_M | 70.1 GB (2 parts) | 82.5 GB | Too big | Get file → | |
| — | 75.2 GB (2 parts) | 87.7 GB | Too big | Get file → |
For 8 GB VRAM use the Q4_K_M quant, which is 70.1 GB and requires about 82.5 GB of RAM at 8 k context. If more memory is available, the unknown quant is 75.24 GB with ~87.7 GB RAM usage. Both are split into two shards.
What GPU do you need for NVIDIA Nemotron 3.5 Super VL 120B A12B?
These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: no quant fits at an 8K context.
- 6 GB VRAM: no quant fits at an 8K context.
- 8 GB VRAM: no quant fits at an 8K context.
- 12 GB VRAM: no quant fits at an 8K context.
- 16 GB VRAM: no quant fits at an 8K context.
- 24 GB VRAM: no quant fits at an 8K context.
- 32 GB VRAM: no quant fits at an 8K context.
- 48 GB VRAM: no quant fits at an 8K context.
Which NVIDIA Nemotron 3.5 Super VL 120B A12B quant should you choose?
Q4_K_M is the usual starting point for NVIDIA Nemotron 3.5 Super VL 120B A12B: it balances file size and quality well. Here it is a 70.1 GB file.
New to quant names? Read GGUF quantization explained.
Specifications
Architecture from the GGUF header
Frequently asked questions about NVIDIA Nemotron 3.5 Super VL 120B A12B
How much VRAM does NVIDIA Nemotron 3.5 Super VL 120B A12B need?
About 82.5 GB at Q4_K_M with an 8K context window. That is the 70.1 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run NVIDIA Nemotron 3.5 Super VL 120B A12B on an 8 GB GPU?
Not at an 8K context. The smallest file needs about 82.5 GB, so you would need a larger GPU or CPU offloading.
What is the file size of NVIDIA Nemotron 3.5 Super VL 120B A12B GGUF?
It depends on the quant: Q4_K_M is 70.1 GB, file is 75.2 GB.
What context length does NVIDIA Nemotron 3.5 Super VL 120B A12B support?
The GGUF header lists a trained context length of 1,048,576 tokens (1024K). The KV cache grows with the context you use: about 11.9 GB at 8K and 47.8 GB at 32K.
Where can I download NVIDIA Nemotron 3.5 Super VL 120B A12B GGUF?
The files are on Hugging Face in the lmstudio-community/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
Can I run NVIDIA Nemotron 3.5 Super VL 120B on 8 GB VRAM?
No, the model requires more than 70 GB of storage and over 80 GB of RAM at 8 k context. It is not suitable for 8 GB VRAM.
Does the model support vision tasks?
Yes, the model has detected vision abilities and can process image inputs.
What quantizations are available?
Two GGUF files are provided: Q4_K_M and an unknown quant, both split into two shards.