QW

Qwen3.5 9B Heretic v2

QwenGGUF by AIImageStudioUpdated 2026-03-05
Download on Hugging Face
Parameters8.95B
Context256K
Layers32
Architectureqwen35
Smallest quant9.5 GB
Quick answer

Qwen3.5 9B Heretic v2 needs about 11.1 GB of memory at Q8_0 (a 9.5 GB file) with an 8K context window, so it fits on a 12 GB GPU. Longer context needs more: at 32K tokens the KV cache alone adds about 4.3 GB.

Qwen3.5 9B Heretic v2 – 9B multimodal model with vision and 262,144 token context

Qwen3.5 9B Heretic v2 is a 9‑billion parameter multimodal model that supports vision and large context. It is a post‑trained version of Qwen3.5, built with gated delta networks and sparse Mixture‑of‑Experts for efficient inference. The model is released under the Apache‑2.0 license.

Good at

  • Vision‑language tasks
  • Large‑context reasoning up to 262,144 tokens
  • Multilingual support across 201 languages

Limits

  • Requires 11.1 GB VRAM at 8 k context (Q8_0 quant)
  • No other quantizations available
  • Model size 9.53 GB in GGUF format

Best for

  • Multimodal chat applications
  • Long‑form document analysis
  • Cross‑lingual content generation

Who it’s for. Developers and researchers needing a large‑context, vision‑enabled language model. The model is suitable for applications that require both text and image understanding within a single inference pipeline.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
Q8_09.5 GB 11.1 GB
Too big Get file →

For systems with 8 GB VRAM, the Q8_0 quantization will exceed memory limits. A 16 GB VRAM setup can run the Q8_0 quant at 8 k context, using approximately 11.1 GB. If more VRAM is available, the same quant can handle larger contexts up to 262,144 tokens.

What GPU do you need for Qwen3.5 9B Heretic v2?

These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: no quant fits at an 8K context.
  • 6 GB VRAM: no quant fits at an 8K context.
  • 8 GB VRAM: no quant fits at an 8K context.
  • 12 GB VRAM: the best quant that fits is Q8_0 (9.5 GB file, about 11.1 GB in use).

Which Qwen3.5 9B Heretic v2 quant should you choose?

Q8_0 (9.5 GB) is the middle option in this repository. Close to lossless. It costs roughly twice the memory of a 4-bit file, so it suits cases where you have plenty of VRAM.

New to quant names? Read GGUF quantization explained.

Specifications

Architecture from the GGUF header

Architectureqwen35
Parameters8.95B
Layers32
Trained context262,144 tokens
KV cache (fp16)128 KB per token

Source

Quantized byAIImageStudio
FamilyQwen
Licenseapache-2.0
Downloads (HF)8,044
Last updated2026-03-05

Frequently asked questions about Qwen3.5 9B Heretic v2

How much VRAM does Qwen3.5 9B Heretic v2 need?

About 11.1 GB at Q8_0 with an 8K context window. That is the 9.5 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run Qwen3.5 9B Heretic v2 on an 8 GB GPU?

Not at an 8K context. The smallest file needs about 11.1 GB, so you would need a larger GPU or CPU offloading.

What is the file size of Qwen3.5 9B Heretic v2 GGUF?

It depends on the quant: Q8_0 is 9.5 GB.

What context length does Qwen3.5 9B Heretic v2 support?

The GGUF header lists a trained context length of 262,144 tokens (256K). The KV cache grows with the context you use: about 1.1 GB at 8K and 4.3 GB at 32K.

Where can I download Qwen3.5 9B Heretic v2 GGUF?

The files are on Hugging Face in the AIImageStudio/Qwen3.5-9b-heretic-v2-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does Qwen3.5 9B Heretic v2 use?

Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run Qwen3.5 9B Heretic v2 on 8 GB VRAM?

No. The Q8_0 quant requires about 11.1 GB VRAM at 8 k context, exceeding an 8 GB GPU.

What license does this model use?

Apache‑2.0. The model can be used freely for commercial and non‑commercial purposes.

Does the model support vision tasks?

Yes. The model includes a vision encoder and is capable of multimodal inference.

Other GGUFs of Qwen3.5-9B-heretic-v2

Similar models

Browse all →

More Qwen models

All Qwen →

Other small models

All small models →