NE

NVIDIA Nemotron Labs Dory1.0 4B

NemotronGGUF by nvidiaUpdated 2026-10-26
Download on Hugging Face
Parameters3.98B
Context256K
Layers72
Architecturedory
Smallest quant3.7 GB
Quick answer

NVIDIA Nemotron Labs Dory1.0 4B needs about 6.6 GB of memory at Q5_K_M (a 3.7 GB file) with an 8K context window, so it fits on a 8 GB GPU. Longer context needs more: at 32K tokens the KV cache alone adds about 9.7 GB.

NVIDIA Nemotron Labs Dory1.0 4B – a looped large language model with elastic inference compute

NVIDIA Nemotron Labs Dory1.0 4B is a 4‑billion‑parameter LLM based on a normalized‑GPT architecture with interleaved hybrid sliding window attention. It uses a shared recurrent block applied multiple times to increase compute per token without increasing parameters. The model is ready for commercial or non‑commercial use.

Good at

  • Elastic inference compute – adjust loop count for accuracy vs speed
  • Token‑efficient generation – fewer tokens needed for a task
  • Memory‑efficient looping – last‑loop KV caching eliminates extra cache overhead

Limits

  • Requires high VRAM – ~6.6 GB at 8 k context for Q5_K_M
  • Large context (262 144 tokens) but no vision support
  • Only Q5_K_M quant available in the GGUF set

Best for

  • Inference tasks needing high compute per token
  • Long‑context reasoning up to 262 144 tokens
  • Applications where token count must be minimized

Who it’s for. Engineers and researchers building LLM inference pipelines on NVIDIA hardware who need a flexible, compute‑efficient model that scales with inference budget.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
Q5_K_M3.7 GB 6.6 GB
Fits Get file →

For 8 k context, the Q5_K_M quant (3.67 GB file) requires about 6.6 GB VRAM. This is the only quant available, so use it if your GPU can accommodate that memory. If you have less VRAM, the model cannot be loaded in its current form.

What GPU do you need for NVIDIA Nemotron Labs Dory1.0 4B?

These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: no quant fits at an 8K context.
  • 6 GB VRAM: no quant fits at an 8K context.
  • 8 GB VRAM: the best quant that fits is Q5_K_M (3.7 GB file, about 6.6 GB in use).

Which NVIDIA Nemotron Labs Dory1.0 4B quant should you choose?

Q5_K_M (3.7 GB) is the middle option in this repository. Closer to the original model than 4-bit, for a moderate increase in size. A good pick when you have memory to spare.

New to quant names? Read GGUF quantization explained.

Specifications

Architecture from the GGUF header

Architecturedory
Parameters3.98B
Layers72
Trained context262,144 tokens
KV cache (fp16)288 KB per token

Source

Quantized bynvidia
FamilyNemotron
Licenseother
Downloads (HF)18,105
Last updated2026-10-26

Frequently asked questions about NVIDIA Nemotron Labs Dory1.0 4B

How much VRAM does NVIDIA Nemotron Labs Dory1.0 4B need?

About 6.6 GB at Q5_K_M with an 8K context window. That is the 3.7 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run NVIDIA Nemotron Labs Dory1.0 4B on an 8 GB GPU?

Yes, with Q5_K_M: the 3.7 GB file needs about 6.6 GB at 8K context. Larger quants in this repository do not fit in 8 GB.

What is the file size of NVIDIA Nemotron Labs Dory1.0 4B GGUF?

It depends on the quant: Q5_K_M is 3.7 GB.

What context length does NVIDIA Nemotron Labs Dory1.0 4B support?

The GGUF header lists a trained context length of 262,144 tokens (256K). The KV cache grows with the context you use: about 2.4 GB at 8K and 9.7 GB at 32K.

Where can I download NVIDIA Nemotron Labs Dory1.0 4B GGUF?

The files are on Hugging Face in the nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does NVIDIA Nemotron Labs Dory1.0 4B use?

Hugging Face lists the license as other. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run this model on 8 GB VRAM?

The Q5_K_M quant needs ~6.6 GB VRAM at 8 k context, leaving little headroom for overhead, so 8 GB may be tight.

Does the model support vision inputs?

No, the model is purely text‑based and does not include vision capabilities.

Is the model suitable for commercial use?

Yes, the model is ready for commercial or non‑commercial use as stated in its documentation.

Similar models

Browse all →

More from nvidia

All by nvidia →

More Nemotron models

All Nemotron →

Other small models

All small models →