Huihui gemma 4 31B it abliterated v2 needs about 17.7 GB of memory at Q3_K_XL (a 15.3 GB file) with an 8K context window, so it fits on a 24 GB GPU. This repository has 8 quants, from 0.0 GB to 35.0 GB (Q8_K_XL).
Huihui gemma 4 31B it abliterated v2 – a reduced‑filtering Gemma 4 model
Huihui gemma 4 31B it abliterated v2 is a 30.7 billion‑parameter Gemma model modified to reduce safety filtering. It is released in GGUF format for llama.cpp and compatible runtimes. The model is licensed under apache‑2.0.
Good at
- Reduced refusal rates compared to original Gemma 4
- Suitable for research and controlled testing
- Supports 8 k context length
Limits
- Limited safety filtering – may generate sensitive content
- No vision or multimodal support
- Requires careful monitoring of outputs
Best for
- Research on safety‑filtering impacts
- Controlled experimentation with large language models
- Testing in low‑latency local inference setups
Who it’s for. Researchers, developers, and testers who need a large Gemma model with reduced safety filters for controlled environments. The model is not recommended for public or production use without additional safeguards.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| — | 0.0 GB | 0.5 GB | Fits | Get file → | |
| IQ2_XXS | 8.6 GB | 10.1 GB | Too big | Get file → | |
| Q2_K_XL | 11.8 GB | 13.7 GB | Too big | Get file → | |
| Q3_K_XL | 15.3 GB | 17.7 GB | Too big | Get file → | |
| Q4_K_XL | 19.2 GB | 22.0 GB | Too big | Get file → | |
| Q5_K_XL | 22.2 GB | 25.3 GB | Too big | Get file → | |
| Q6_K_XL | 27.5 GB | 31.3 GB | Too big | Get file → | |
| Q8_K_XL | 35.0 GB | 39.7 GB | Too big | Get file → |
For 8 GB VRAM, use IQ2_XXS (file size 8.59 GB, estimated 10.1 GB memory). For 16 GB VRAM, Q3_K_XL (15.32 GB, 17.7 GB memory) is suitable. Larger VRAM allows Q4_K_XL or Q5_K_XL for higher precision.
What GPU do you need for Huihui gemma 4 31B it abliterated v2?
These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
- 6 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
- 8 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
- 12 GB VRAM: the best quant that fits is IQ2_XXS (8.6 GB file, about 10.1 GB in use).
- 16 GB VRAM: the best quant that fits is Q2_K_XL (11.8 GB file, about 13.7 GB in use).
- 24 GB VRAM: the best quant that fits is Q4_K_XL (19.2 GB file, about 22.0 GB in use).
- 32 GB VRAM: the best quant that fits is Q6_K_XL (27.5 GB file, about 31.3 GB in use).
- 48 GB VRAM: the best quant that fits is Q8_K_XL (35.0 GB file, about 39.7 GB in use).
Which Huihui gemma 4 31B it abliterated v2 quant should you choose?
Q3_K_XL (15.3 GB) is the middle option in this repository. A reasonable choice when a 4-bit file will not fit, mostly on larger models. Quality loss is noticeable but often acceptable.
If memory is tight, the smallest file is the first one in the table at 0.0 GB, with the quality trade-offs that come with fewer bits.
New to quant names? Read GGUF quantization explained.
Specifications
Architecture from the GGUF header
Frequently asked questions about Huihui gemma 4 31B it abliterated v2
How much VRAM does Huihui gemma 4 31B it abliterated v2 need?
About 17.7 GB at Q3_K_XL with an 8K context window. That is the 15.3 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run Huihui gemma 4 31B it abliterated v2 on an 8 GB GPU?
Yes, with the smaller files: the 0.0 GB file needs about 0.5 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of Huihui gemma 4 31B it abliterated v2 GGUF?
It depends on the quant: file is 0.0 GB, IQ2_XXS is 8.6 GB, Q2_K_XL is 11.8 GB, Q3_K_XL is 15.3 GB, Q4_K_XL is 19.2 GB, Q5_K_XL is 22.2 GB and 2 more.
Where can I download Huihui gemma 4 31B it abliterated v2 GGUF?
The files are on Hugging Face in the groxaxo/Huihui-gemma-4-31B-it-abliterated-v2-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does Huihui gemma 4 31B it abliterated v2 use?
Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run this model on 8 GB VRAM?
Yes, the IQ2_XXS quantization requires about 10.1 GB memory at 8 k context, which fits within 8 GB VRAM with some overhead.
Does this model support vision?
No, the model is purely text and does not include vision or multimodal capabilities.
What license applies to this model?
The model is released under the apache‑2.0 license.