NE

IQ2 NL IQ3 NL Test Models

NemotronGGUF by eaddarioUpdated 2026-08-18
Download on Hugging Face
Parameters8.89B
Context1024K
Layers—
Architecturenemotron_h
Smallest quant0.0 GB
Quick answer

IQ2 NL IQ3 NL Test Models needs about 7.8 GB of memory at Q4_K (a 6.5 GB file) with an 8K context window, so it fits on a 8 GB GPU. This repository has 17 quants, from 0.0 GB to 17.8 GB (F16).

IQ2 NL IQ3 NL Test Models – Quantized variants of NVIDIA‑Nemotron‑Nano‑9B‑v2

IQ2 NL IQ3 NL Test Models are experimental quantizations of NVIDIA‑Nemotron‑Nano‑9B‑v2. They use the nemotron_h hybrid Mamba‑2 / attention architecture and support 1,048,576 context tokens. The models are released under the MIT license.

Good at

  • Hybrid Mamba‑2 / attention architecture
  • Large context support
  • Experimental IQ2_NL and IQ3_NL quantization

Limits

  • No vision support
  • Requires llama.cpp fork for IQ2_NL/IQ3_NL
  • Large memory for higher‑bit quants

Best for

  • Chat and reasoning tasks
  • Testing low‑bit quantization
  • Research on non‑linear block types

Who it’s for. Developers and researchers interested in evaluating new quantization types for the Nemotron family. The models are suitable for those who can run llama.cpp with the required fork and are comfortable with experimental releases.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
—0.0 GB 0.5 GB
Fits Get file →
IQ2_NL3.1 GB 4.0 GB
Fits Get file →
Q2_03.2 GB 4.1 GB
Fits Get file →
IQ1_M3.4 GB 4.3 GB
Fits Get file →
IQ3_NL4.1 GB 5.1 GB
Fits Get file →
IQ1_M4.9 GB 5.9 GB
Fits Get file →
Q2_K5.0 GB 6.1 GB
Fits Get file →
Q3_K5.4 GB 6.5 GB
Fits Get file →
Q4_K6.5 GB 7.8 GB
Tight Get file →
Q4_K6.5 GB 7.8 GB
Tight Get file →
Q5_K7.1 GB 8.4 GB
Too big Get file →
Q5_K7.1 GB 8.4 GB
Too big Get file →
Q6_K9.1 GB 10.7 GB
Too big Get file →
Q6_K9.1 GB 10.7 GB
Too big Get file →
Q8_09.5 GB 11.1 GB
Too big Get file →
Q8_09.5 GB 11.1 GB
Too big Get file →
F1617.8 GB 20.4 GB
Too big Get file →

For 8 GB VRAM, use IQ2_NL (file size 3.12 GB, memory 4 GB at 8 k context). For 12 GB VRAM, Q4_K (file size 6.53 GB, memory 7.8 GB). For 20 GB VRAM, F16 (file size 17.79 GB, memory 20.4 GB). The estimates are approximate.

What GPU do you need for IQ2 NL IQ3 NL Test Models?

These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: the best quant that fits is IQ2_NL (3.1 GB file, about 4.0 GB in use).
  • 6 GB VRAM: the best quant that fits is IQ1_M (4.9 GB file, about 5.9 GB in use).
  • 8 GB VRAM: the best quant that fits is Q4_K (6.5 GB file, about 7.8 GB in use).
  • 12 GB VRAM: the best quant that fits is Q8_0 (9.5 GB file, about 11.1 GB in use).
  • 16 GB VRAM: the best quant that fits is Q8_0 (9.5 GB file, about 11.1 GB in use).
  • 24 GB VRAM: the best quant that fits is F16 (17.8 GB file, about 20.4 GB in use).

Which IQ2 NL IQ3 NL Test Models quant should you choose?

Q4_K (6.5 GB) is the middle option in this repository. The usual sweet spot between file size and quality, and the quant most people start with.

If memory is tight, the smallest file is the first one in the table at 0.0 GB, with the quality trade-offs that come with fewer bits.

New to quant names? Read GGUF quantization explained.

Specifications

Architecture from the GGUF header

Architecturenemotron_h
Parameters8.89B
Trained context1,048,576 tokens

Source

Quantized byeaddario
FamilyNemotron
Licensemit
Downloads (HF)744
Last updated2026-08-18

Frequently asked questions about IQ2 NL IQ3 NL Test Models

How much VRAM does IQ2 NL IQ3 NL Test Models need?

About 7.8 GB at Q4_K with an 8K context window. That is the 6.5 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run IQ2 NL IQ3 NL Test Models on an 8 GB GPU?

Yes, with Q4_K: the 6.5 GB file needs about 7.8 GB at 8K context. Larger quants in this repository do not fit in 8 GB.

What is the file size of IQ2 NL IQ3 NL Test Models GGUF?

It depends on the quant: file is 0.0 GB, IQ2_NL is 3.1 GB, Q2_0 is 3.2 GB, IQ1_M is 3.4 GB, IQ3_NL is 4.1 GB, IQ1_M is 4.9 GB and 11 more.

What context length does IQ2 NL IQ3 NL Test Models support?

The GGUF header lists a trained context length of 1,048,576 tokens (1024K).

Where can I download IQ2 NL IQ3 NL Test Models GGUF?

The files are on Hugging Face in the eaddario/IQ2_NL-IQ3_NL-Test-Models repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does IQ2 NL IQ3 NL Test Models use?

Hugging Face lists the license as mit. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run IQ2_NL on 8 GB VRAM?

Yes. The IQ2_NL quant is 3.12 GB and requires about 4 GB of VRAM at 8 k context.

Do these models support vision?

No. The models are purely language models with no vision capability.

What license applies to these models?

They are released under the MIT license.

Other GGUFs of NVIDIA-Nemotron-Nano-9B-v2

Similar models

Browse all →

More from eaddario

All by eaddario →

More Nemotron models

All Nemotron →

Other small models

All small models →