Hy3 295B A21B needs about 131.2 GB of memory at its main file (a 116.7 GB file) with an 8K context window, which is more than a 48 GB GPU holds. This repository has 3 quants, from 0.6 GB to 143.9 GB.
Hy3 295B A21B – 295B parameter model with 21B active MoE
Hy3 295B A21B is a 295B parameter Mixture‑of‑Experts model with 21B active parameters. It is released under Apache‑2.0 and is available in GGUF format for llama.cpp. The model supports large context lengths and is optimized for inference on high‑memory GPUs.
Good at
- Large‑scale reasoning
- Agentic task performance
- Long‑context handling
Limits
- No vision support
- Requires high VRAM for full model
- No quantization options listed
Best for
- Enterprise coding assistants
- Financial modeling tools
- Game development pipelines
Who it’s for. Developers and product teams needing a high‑capacity language model for complex reasoning and agentic workflows. The model is suitable for environments with 128 GB GPU memory or equivalent unified RAM.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| — | 0.6 GB | 1.2 GB | Fits | Get file → | |
| — | 116.7 GB (3 parts) | 131.2 GB | Too big | Get file → | |
| — | 143.9 GB (4 parts) | 161.6 GB | Too big | Get file → |
For 128 GB GPU memory, use the 116.67 GB file (est. 131.2 GB memory at 8K context). If memory is limited to 107 GB, the 143.87 GB file (est. 161.6 GB memory) may still fit on 128 GB systems but will use more RAM. The 0.6 GB file is a placeholder and not a usable quantization.
What GPU do you need for Hy3 295B A21B?
These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: the best quant that fits is the main file (0.6 GB file, about 1.2 GB in use).
- 6 GB VRAM: the best quant that fits is the main file (0.6 GB file, about 1.2 GB in use).
- 8 GB VRAM: the best quant that fits is the main file (0.6 GB file, about 1.2 GB in use).
- 12 GB VRAM: the best quant that fits is the main file (0.6 GB file, about 1.2 GB in use).
- 16 GB VRAM: the best quant that fits is the main file (0.6 GB file, about 1.2 GB in use).
- 24 GB VRAM: the best quant that fits is the main file (0.6 GB file, about 1.2 GB in use).
- 32 GB VRAM: the best quant that fits is the main file (0.6 GB file, about 1.2 GB in use).
- 48 GB VRAM: the best quant that fits is the main file (0.6 GB file, about 1.2 GB in use).
Which Hy3 295B A21B quant should you choose?
If memory is tight, the smallest file is the first one in the table at 0.6 GB, with the quality trade-offs that come with fewer bits.
New to quant names? Read GGUF quantization explained.
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about Hy3 295B A21B
How much VRAM does Hy3 295B A21B need?
About 131.2 GB at its main file with an 8K context window. That is the 116.7 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run Hy3 295B A21B on an 8 GB GPU?
Yes, with the smaller files: the 0.6 GB file needs about 1.2 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of Hy3 295B A21B GGUF?
It depends on the quant: file is 0.6 GB, file is 116.7 GB, file is 143.9 GB.
Where can I download Hy3 295B A21B GGUF?
The files are on Hugging Face in the YanissAmz/Hy3-295B-A21B-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does Hy3 295B A21B use?
Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run Hy3 295B A21B on a 128 GB GPU?
Yes, the 116.67 GB GGUF file fits within 128 GB GPU memory, with an estimated resident footprint of ~107 GB.
Does the model support vision tasks?
No, Hy3 295B A21B does not include vision capabilities.
What license applies to this model?
The model is released under the Apache‑2.0 license.