QW

MiMo V2.6 Distill Qwen 9B

QwenGGUF by VitreousCutUpdated 2026-09-24
Download on Hugging Face
Parameters9.2B
Context256K
Layers—
Architectureqwen35
Smallest quant0.0 GB
Quick answer

MiMo V2.6 Distill Qwen 9B needs about 7.3 GB of memory at Q4_K_M (a 6.1 GB file) with an 8K context window, so it fits on a 8 GB GPU. This repository has 3 quants, from 0.0 GB to 6.7 GB (Q5_K_M).

MiMo V2.6 Distill Qwen 9B – a 9B agentic model with multi‑token‑prediction

MiMo V2.6 Distill Qwen 9B is a 9.2B parameter model trained by Xiaomi MiMo on Qwen3.5‑9B. It supports coding, general‑purpose agent tasks, visual coding, and cybersecurity. The model is released under the MIT license and includes a multi‑token‑prediction head for speculative decoding.

Good at

  • Coding tasks
  • Agentic reasoning
  • Multi‑token speculative decoding

Limits

  • No vision support
  • Requires Qwen3.5‑compatible llama.cpp build
  • Large VRAM needed for higher quantisations

Best for

  • Developers building agentic applications
  • Researchers testing speculative decoding
  • Code generation projects

Who it’s for. This model is suitable for developers and researchers who need a 9B agentic model with a multi‑token‑prediction head. It is ideal for those working on coding, agentic tasks, and cybersecurity applications who can run it on a compatible llama.cpp build.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
—0.0 GB 0.5 GB
Fits Get file →
Q4_K_M6.1 GB 7.3 GB
Tight Get file →
Q5_K_M6.7 GB 8.0 GB
Too big Get file →

For 8 GB VRAM, use the Q4_K_M quant (6.1 GB file, ~7.3 GB memory at 8 k context). For 12 GB VRAM, Q5_K_M (6.73 GB file, ~8 GB memory) is recommended. The unknown quant file is only 0.01 GB but memory usage is not specified.

What GPU do you need for MiMo V2.6 Distill Qwen 9B?

These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
  • 6 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
  • 8 GB VRAM: the best quant that fits is Q5_K_M (6.7 GB file, about 8.0 GB in use).

Which MiMo V2.6 Distill Qwen 9B quant should you choose?

Q4_K_M is the usual starting point for MiMo V2.6 Distill Qwen 9B: it balances file size and quality well. Here it is a 6.1 GB file.

If memory is tight, the smallest file is the first one in the table at 0.0 GB, with the quality trade-offs that come with fewer bits.

New to quant names? Read GGUF quantization explained.

Specifications

Architecture from the GGUF header

Architectureqwen35
Parameters9.2B
Trained context262,144 tokens

Source

Quantized byVitreousCut
FamilyQwen
Licensemit
Downloads (HF)1,391
Last updated2026-09-24

Frequently asked questions about MiMo V2.6 Distill Qwen 9B

How much VRAM does MiMo V2.6 Distill Qwen 9B need?

About 7.3 GB at Q4_K_M with an 8K context window. That is the 6.1 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run MiMo V2.6 Distill Qwen 9B on an 8 GB GPU?

Yes, with Q5_K_M: the 6.7 GB file needs about 8.0 GB at 8K context. Larger quants in this repository do not fit in 8 GB.

What is the file size of MiMo V2.6 Distill Qwen 9B GGUF?

It depends on the quant: file is 0.0 GB, Q4_K_M is 6.1 GB, Q5_K_M is 6.7 GB.

What context length does MiMo V2.6 Distill Qwen 9B support?

The GGUF header lists a trained context length of 262,144 tokens (256K).

Where can I download MiMo V2.6 Distill Qwen 9B GGUF?

The files are on Hugging Face in the VitreousCut/MiMo-V2.6-Distill-Qwen-9B-MTP-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does MiMo V2.6 Distill Qwen 9B use?

Hugging Face lists the license as mit. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run MiMo V2.6 Distill Qwen 9B on 8 GB VRAM?

Yes, using the Q4_K_M quant will use about 7.3 GB of VRAM at 8 k context, which fits within 8 GB. The Q5_K_M quant would exceed 8 GB.

Does this model support vision tasks?

No, the model does not include vision capabilities.

What license does the model use?

The model is released under the MIT license.

Other GGUFs of MiMo-V2.6-Distill-Qwen-9B

Similar models

Browse all →

More from VitreousCut

All by VitreousCut →

More Qwen models

All Qwen →

Other small models

All small models →