Mistral Small 4 119B ROCmFP4 needs about 68.6 GB of memory at Q4_0 (a 67.7 GB file) with an 8K context window, which is more than a 48 GB GPU holds. This repository has 2 quants, from 67.7 GB (Q4_0) to 122.8 GB (Q8_0). Longer context needs more: at 32K tokens the KV cache alone adds about 1.4 GB.
Mistral Small 4 119B ROCmFP4 – 119 B parameter MoE model with 256k context
Mistral Small 4 119B ROCmFP4 is a 119 B parameter mixture‑of‑experts model built on the mistral4 architecture. It supports a 256k token context window and multimodal input, delivering instruction and reasoning capabilities. The model is released under Apache 2.0 and is quantized for AMD ROCmFP4 hardware.
Good at
- Large context handling
- Multimodal text‑image input
- Reasoning and instruction tasks
Limits
- Requires ROCmFP4‑capable llama.cpp
- Large VRAM needed for 8‑bit build
- No native vision support on non‑ROCm hardware
Best for
- Long‑form content generation
- Complex reasoning workflows
- Multilingual instruction following
Who it’s for. Developers building high‑capacity LLM services on AMD GPUs. Researchers needing a large MoE model with extensive context and multimodal support. Teams requiring an open‑source model under Apache 2.0 for commercial use.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| Q4_0 | 67.7 GB (2 parts) | 68.6 GB | Too big | Get file → | |
| Q8_0 | 122.8 GB | 123.7 GB | Too big | Get file → |
For 8 GB VRAM use the Q4_0 build (67.73 GB file, ~68.6 GB memory at 8 k context). For 16 GB VRAM the Q8_0 build (122.82 GB file, ~123.7 GB memory at 8 k context) is recommended. Memory estimates are approximate.
What GPU do you need for Mistral Small 4 119B ROCmFP4?
These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: no quant fits at an 8K context.
- 6 GB VRAM: no quant fits at an 8K context.
- 8 GB VRAM: no quant fits at an 8K context.
- 12 GB VRAM: no quant fits at an 8K context.
- 16 GB VRAM: no quant fits at an 8K context.
- 24 GB VRAM: no quant fits at an 8K context.
- 32 GB VRAM: no quant fits at an 8K context.
- 48 GB VRAM: no quant fits at an 8K context.
Which Mistral Small 4 119B ROCmFP4 quant should you choose?
Q4_0 (67.7 GB) is the middle option in this repository. The usual sweet spot between file size and quality, and the quant most people start with.
New to quant names? Read GGUF quantization explained.
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about Mistral Small 4 119B ROCmFP4
How much VRAM does Mistral Small 4 119B ROCmFP4 need?
About 68.6 GB at Q4_0 with an 8K context window. That is the 67.7 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run Mistral Small 4 119B ROCmFP4 on an 8 GB GPU?
Not at an 8K context. The smallest file needs about 68.6 GB, so you would need a larger GPU or CPU offloading.
What is the file size of Mistral Small 4 119B ROCmFP4 GGUF?
It depends on the quant: Q4_0 is 67.7 GB, Q8_0 is 122.8 GB.
What context length does Mistral Small 4 119B ROCmFP4 support?
The GGUF header lists a trained context length of 1,048,576 tokens (1024K). The KV cache grows with the context you use: about 0.3 GB at 8K and 1.4 GB at 32K.
Where can I download Mistral Small 4 119B ROCmFP4 GGUF?
The files are on Hugging Face in the kingjones777/Mistral-Small-4-119B-ROCmFP4-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does Mistral Small 4 119B ROCmFP4 use?
Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run Mistral Small 4 119B ROCmFP4 on 8 GB VRAM?
The Q4_0 quantized model requires about 68.6 GB of VRAM at 8 k context, so 8 GB is insufficient.
Does this model support vision tasks?
Yes, it accepts image input and provides text output, but it requires ROCmFP4‑capable hardware.
What license applies to this model?
The model is released under the Apache 2.0 license, allowing commercial and non‑commercial use.