Ingot XVI 27B Q2 needs about 8.6 GB of memory at its main file (a 5.9 GB file) with an 8K context window, so it fits on a 12 GB GPU. This repository has 2 quants, from 5.9 GB to 7.2 GB. Longer context needs more: at 32K tokens the KV cache alone adds about 8.6 GB.
Ingot XVI 27B Q2 – ternary Qwen3.8‑27B model for llama.cpp
Ingot XVI 27B Q2 is a 26.9B‑parameter ternary model based on Qwen3.8‑27B. It offers 262,144‑token context and retains 98.2% of FP16 intelligence. Two GGUF packs are available: PTQ1_0 (5.95 GB, 8.6 GB memory at 8 k context) and PQ2_0 (7.21 GB, 9.9 GB memory at 8 k context).
Good at
- 262K‑token context
- reasoning and tool use
- ternary weights for efficient inference
Limits
- no vision support
- requires GPU or CPU with llama.cpp
- memory ~8–10 GB at 8 k context
Best for
- long‑form reasoning
- tool‑calling agents
- in‑device inference on laptops
Who it’s for. Developers and researchers who need large‑context reasoning on a single GPU or CPU. The model runs on llama.cpp with CUDA, Metal, or CPU backends and is suitable for on‑device inference without a vision tower.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| — | 5.9 GB | 8.6 GB | Too big | Get file → | |
| — | 7.2 GB | 9.9 GB | Too big | Get file → |
For 8 GB VRAM, use the PTQ1_0 pack (5.95 GB file, ~8.6 GB memory). If 10 GB VRAM is available, the PQ2_0 pack (7.21 GB file, ~9.9 GB memory) provides a slightly larger representation. Both packs keep ternary weights and never expand to FP16.
What GPU do you need for Ingot XVI 27B Q2?
These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: no quant fits at an 8K context.
- 6 GB VRAM: no quant fits at an 8K context.
- 8 GB VRAM: no quant fits at an 8K context.
- 12 GB VRAM: the best quant that fits is the main file (7.2 GB file, about 9.9 GB in use).
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about Ingot XVI 27B Q2
How much VRAM does Ingot XVI 27B Q2 need?
About 8.6 GB at its main file with an 8K context window. That is the 5.9 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run Ingot XVI 27B Q2 on an 8 GB GPU?
Not at an 8K context. The smallest file needs about 8.6 GB, so you would need a larger GPU or CPU offloading.
What is the file size of Ingot XVI 27B Q2 GGUF?
It depends on the quant: file is 5.9 GB, file is 7.2 GB.
What context length does Ingot XVI 27B Q2 support?
The GGUF header lists a trained context length of 262,144 tokens (256K). The KV cache grows with the context you use: about 2.1 GB at 8K and 8.6 GB at 32K.
Where can I download Ingot XVI 27B Q2 GGUF?
The files are on Hugging Face in the dendriteholdings/Ingot-XVI-27B-Q2 repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
Can I run Ingot XVI 27B Q2 on 8 GB VRAM?
Yes, the PTQ1_0 pack requires about 8.6 GB memory at 8 k context, which fits on 8 GB VRAM with some headroom.
Does this model support vision tasks?
No, the vision tower is a separate Q8_0 mmproj pack and is not included in the GGUF files.
What context length does it support?
It supports 262,144 tokens, enabling very long‑form text processing on device.