Qwen3.6 27B Ablit IQ4 XS needs about 0.5 GB of memory at its main file (a 0.0 GB file) with an 8K context window, so it fits on a 4 GB GPU. This repository has 2 quants, from 0.0 GB to 15.1 GB (IQ4_XS).
Qwen3.6 27B Ablit IQ4 XS – a 26.9B parameter local LLM
This model is a 26.9B parameter Qwen variant that has undergone abliteration. It is available in a small IQ4_XS quantized GGUF format. The model supports a 262144 token context window and is designed for local inference with llama.cpp.
Good at
- Large context handling
- Abliterated refusal control
- ChatML compatibility
Limits
- No vision support
- Requires 17.4GB VRAM at 8k context
- License not specified
Best for
- Chat applications
- Long‑form content generation
- Tool‑based agentic tasks
Who it’s for. Users who need a large‑parameter LLM that can run locally on consumer GPUs. The IQ4_XS quant is the recommended format for those with limited VRAM but still wanting a 26.9B model.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| — | 0.0 GB | 0.5 GB | Fits | Get file → | |
| IQ4_XS | 15.1 GB | 17.4 GB | Too big | Get file → |
The IQ4_XS quant is 15.08GB and uses about 17.4GB of VRAM at 8k context, making it suitable for GPUs with 16GB+ VRAM. A smaller unknown quant is 0.01GB but its memory usage is 0.5GB at 8k context, which may fit on very low‑end hardware but is not recommended for full‑scale use.
What GPU do you need for Qwen3.6 27B Ablit IQ4 XS?
These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
- 6 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
- 8 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
- 12 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
- 16 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
- 24 GB VRAM: the best quant that fits is IQ4_XS (15.1 GB file, about 17.4 GB in use).
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about Qwen3.6 27B Ablit IQ4 XS
How much VRAM does Qwen3.6 27B Ablit IQ4 XS need?
About 0.5 GB at its main file with an 8K context window. That is the 0.0 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run Qwen3.6 27B Ablit IQ4 XS on an 8 GB GPU?
Yes, with the smaller files: the 0.0 GB file needs about 0.5 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of Qwen3.6 27B Ablit IQ4 XS GGUF?
It depends on the quant: file is 0.0 GB, IQ4_XS is 15.1 GB.
What context length does Qwen3.6 27B Ablit IQ4 XS support?
The GGUF header lists a trained context length of 262,144 tokens (256K).
Where can I download Qwen3.6 27B Ablit IQ4 XS GGUF?
The files are on Hugging Face in the sci4ai/Qwen3.6-27B-Ablit-IQ4_XS-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
Can I run this model on 8 GB VRAM?
No. The IQ4_XS quant requires about 17.4GB of VRAM at 8k context.
Does this model support vision?
No, it does not have vision capabilities.
What is abliteration?
Abliteration removes specific refusal directions from the model to reduce unwanted refusals.