Gemma4 needs about 20.4 GB of memory at its main file (a 7.0 GB file) with an 8K context window, so it fits on a 24 GB GPU. Longer context needs more: at 32K tokens the KV cache alone adds about 51.5 GB.
Gemma4 12B uncensored multimodal model in NVFP4 GGUF format
Gemma4 12B uncensored is a BF16 → NVFP4 quantized GGUF model. It retains Gemma 4’s encoder‑free unified multimodal stack, supporting text, image, and audio inputs. The checkpoint is a drop‑in replacement for the original instruction‑tuned model.
Good at
- Multimodal text, image, audio
- Uncensored instruction tuning
- Drop‑in compatibility with gemma-4-12B-it
Limits
- Experimental quantization, no calibration
- Requires ~20.4 GB VRAM at 8 k context
- Large context size (262,144 tokens) may need more VRAM
Best for
- Local multimodal inference
- Fine‑tuning on custom datasets
- Running alongside other models in shared VRAM
Who it’s for. Developers and researchers needing a local, multimodal LLM that can process text, images, and audio without separate vision or audio towers. The model is suitable for applications requiring uncensored responses and reasoning capabilities.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| — | 7.0 GB | 20.4 GB | Too big | Get file → |
The only available GGUF is NVFP4 with a file size of 6.98 GB. It estimates 20.4 GB VRAM at 8 k context. For 8 k context, this is the recommended quant. No other quant variants are listed.
What GPU do you need for Gemma4?
These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: no quant fits at an 8K context.
- 6 GB VRAM: no quant fits at an 8K context.
- 8 GB VRAM: no quant fits at an 8K context.
- 12 GB VRAM: no quant fits at an 8K context.
- 16 GB VRAM: no quant fits at an 8K context.
- 24 GB VRAM: the best quant that fits is the main file (7.0 GB file, about 20.4 GB in use).
Specifications
Architecture from the GGUF header
Frequently asked questions about Gemma4
How much VRAM does Gemma4 need?
About 20.4 GB at its main file with an 8K context window. That is the 7.0 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run Gemma4 on an 8 GB GPU?
Not at an 8K context. The smallest file needs about 20.4 GB, so you would need a larger GPU or CPU offloading.
What is the file size of Gemma4 GGUF?
It depends on the quant: file is 7.0 GB.
What context length does Gemma4 support?
The GGUF header lists a trained context length of 262,144 tokens (256K). The KV cache grows with the context you use: about 12.9 GB at 8K and 51.5 GB at 32K.
Where can I download Gemma4 GGUF?
The files are on Hugging Face in the Magnarks/gemma-4-12B-it-abliterated-uncensored-nvfp4 repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does Gemma4 use?
Hugging Face lists the license as gemma. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run this model on 8 GB VRAM?
No. The NVFP4 GGUF requires approximately 20.4 GB VRAM at 8 k context. 8 GB VRAM is insufficient.
Is this model uncensored?
Yes. The model is an uncensored variant of Gemma 4 12B, with refusal ablation removed.
Does it support vision?
Yes. The model retains Gemma 4’s encoder‑free unified multimodal stack, handling text, image, and audio inputs.