Millie 1.1 35B A3B Ternary needs about 0.8 GB of memory at its main file (a 0.3 GB file) with an 8K context window, so it fits on a 4 GB GPU. This repository has 2 quants, from 0.3 GB to 8.3 GB.
Millie 1.1 35B A3B Ternary – a compact vision‑enabled LLM
Millie 1.1 35B A3B Ternary is a 35 billion‑parameter language and vision model built for local inference on memory‑constrained devices. Roughly 90 % of its weights are ternary, with the rest in higher precision. It inherits Qwen3.6‑35B‑A3B architecture and supports a 262,144‑token native context length.
Good at
- Vision‑enabled text generation
- Low‑memory inference on local devices
- High‑parameter efficiency with ternary weights
Limits
- Requires 8 k context for memory estimates
- No vision support in the base file
- Large VRAM needed for full model
Best for
- Local chatbots with image input
- Edge inference on mobile GPUs
- Rapid prototyping of vision‑language tasks
Who it’s for. Developers needing a large vision‑language model that runs on devices with limited memory. The model is ideal for local inference scenarios where internet access is unavailable or latency must be minimal.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| — | 0.3 GB | 0.8 GB | Fits | Get file → | |
| — | 8.3 GB | 9.8 GB | Too big | Get file → |
For 8 GB VRAM, use the 0.3 GB file which estimates 0.8 GB RAM at 8 k context. For 16 GB VRAM, the 8.31 GB file uses about 9.8 GB RAM at 8 k context. These estimates are approximate and depend on runtime configuration.
What GPU do you need for Millie 1.1 35B A3B Ternary?
These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: the best quant that fits is the main file (0.3 GB file, about 0.8 GB in use).
- 6 GB VRAM: the best quant that fits is the main file (0.3 GB file, about 0.8 GB in use).
- 8 GB VRAM: the best quant that fits is the main file (0.3 GB file, about 0.8 GB in use).
- 12 GB VRAM: the best quant that fits is the main file (8.3 GB file, about 9.8 GB in use).
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about Millie 1.1 35B A3B Ternary
How much VRAM does Millie 1.1 35B A3B Ternary need?
About 0.8 GB at its main file with an 8K context window. That is the 0.3 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run Millie 1.1 35B A3B Ternary on an 8 GB GPU?
Yes, with the smaller files: the 0.3 GB file needs about 0.8 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of Millie 1.1 35B A3B Ternary GGUF?
It depends on the quant: file is 0.3 GB, file is 8.3 GB.
Where can I download Millie 1.1 35B A3B Ternary GGUF?
The files are on Hugging Face in the llmsforall/Millie-1.1-35B-A3B-Ternary repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does Millie 1.1 35B A3B Ternary use?
Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run Millie 1.1 on 8 GB VRAM?
Yes, the 0.3 GB quantized file uses roughly 0.8 GB RAM at 8 k context, fitting within 8 GB VRAM.
Does the model support vision?
Yes, but the vision file is optional. Download the separate vision gguf to enable image input.
What license applies to this model?
The model is licensed under Apache 2.0, matching the upstream Qwen3.6‑35B‑A3B license.