Mistral_Nemo_Instruct_2407 needs about 26.3 GB of memory at F16 (a 24.5 GB file) with an 8K context window, so it fits on a 32 GB GPU. Longer context needs more: at 32K tokens the KV cache alone adds about 5.4 GB.
12.2B‑parameter Mistral‑Nemo‑Instruct with 128k context
Mistral_Nemo_Instruct_2407 is a 12.2B‑parameter LLM based on the Mistral architecture. It supports a 131072‑token context window and is fine‑tuned for instruction following. The model is released under the Apache‑2.0 license.
Good at
- Instruction following
- Tool calling
- Large context handling
Limits
- No vision support
- Requires high VRAM for full precision
- No multilingual language list provided
Best for
- Chatbots needing long context
- Tool‑enabled assistants
- Code generation with context
Who it’s for. Developers building instruction‑driven applications that require a large context window and tool integration. The model is suitable for users who can provide the necessary GPU memory for full‑precision inference.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| F16 | 24.5 GB | 26.3 GB | Too big | Get file → |
The only available GGUF build is F16, 24.5 GB on disk. It requires approximately 26.3 GB of VRAM at an 8 k context length. For lower memory budgets, no quantized variants are listed, so the model cannot be run within those limits.
What GPU do you need for Mistral_Nemo_Instruct_2407?
These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: no quant fits at an 8K context.
- 6 GB VRAM: no quant fits at an 8K context.
- 8 GB VRAM: no quant fits at an 8K context.
- 12 GB VRAM: no quant fits at an 8K context.
- 16 GB VRAM: no quant fits at an 8K context.
- 24 GB VRAM: no quant fits at an 8K context.
- 32 GB VRAM: the best quant that fits is F16 (24.5 GB file, about 26.3 GB in use).
Which Mistral_Nemo_Instruct_2407 quant should you choose?
F16 (24.5 GB) is the middle option in this repository. The unquantized model. It gives original quality and needs the most memory, so it is mainly for conversion or reference.
New to quant names? Read GGUF quantization explained.
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about Mistral_Nemo_Instruct_2407
How much VRAM does Mistral_Nemo_Instruct_2407 need?
About 26.3 GB at F16 with an 8K context window. That is the 24.5 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run Mistral_Nemo_Instruct_2407 on an 8 GB GPU?
Not at an 8K context. The smallest file needs about 26.3 GB, so you would need a larger GPU or CPU offloading.
What is the file size of Mistral_Nemo_Instruct_2407 GGUF?
It depends on the quant: F16 is 24.5 GB.
What context length does Mistral_Nemo_Instruct_2407 support?
The GGUF header lists a trained context length of 131,072 tokens (128K). The KV cache grows with the context you use: about 1.3 GB at 8K and 5.4 GB at 32K.
Where can I download Mistral_Nemo_Instruct_2407 GGUF?
The files are on Hugging Face in the pbhappliedsystems/mistral-nemo-instruct-2407-gguf-F16 repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does Mistral_Nemo_Instruct_2407 use?
Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run Mistral_Nemo_Instruct_2407 on 8 GB VRAM?
No. The F16 build needs about 26.3 GB of VRAM at 8 k context, which exceeds 8 GB.
What license does this model use?
Apache‑2.0, permitting commercial use.
Does the model support vision or audio?
No, the model has no vision or audio capabilities.