voxtral mini 4b realtime needs about 5.8 GB of memory at Q8_0 (a 4.7 GB file) with an 8K context window, so it fits on a 6 GB GPU. This repository has 3 quants, from 2.5 GB (Q4_K) to 8.9 GB (F16).
4.4B‑parameter realtime speech‑to‑text model
Voxtral Mini 4B Realtime is a 4.43‑billion‑parameter model designed for low‑latency audio transcription. It uses a causal audio encoder and a 26‑layer Mistral decoder. The model supports 13 languages and is available under Apache 2.0.
Good at
- Realtime speech‑to‑text
- Low‑latency transcription
- Multilingual support
Limits
- No vision support
- Requires vLLM for production use
- Maximum model length limited by RoPE pre‑allocations
Best for
- Live podcast transcription
- Real‑time captioning
- Audio‑driven chatbots
Who it’s for. Developers building real‑time speech‑to‑text applications who need a lightweight, open‑source model. The model is suitable for servers with moderate GPU memory and for users who prefer a causal encoder for streaming.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| Q4_K | 2.5 GB | 3.3 GB | Fits | Get file → | |
| Q8_0 | 4.7 GB | 5.8 GB | Fits | Get file → | |
| F16 | 8.9 GB | 10.4 GB | Too big | Get file → |
For 8 GB VRAM use Q4_K (file size 2.52 GB, estimated memory 3.3 GB at 8k context). For 12 GB VRAM Q8_0 (4.73 GB, 5.8 GB). F16 requires 8.88 GB and 10.4 GB memory, suitable for high‑end GPUs.
What GPU do you need for voxtral mini 4b realtime?
These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: the best quant that fits is Q4_K (2.5 GB file, about 3.3 GB in use).
- 6 GB VRAM: the best quant that fits is Q8_0 (4.7 GB file, about 5.8 GB in use).
- 8 GB VRAM: the best quant that fits is Q8_0 (4.7 GB file, about 5.8 GB in use).
- 12 GB VRAM: the best quant that fits is F16 (8.9 GB file, about 10.4 GB in use).
Which voxtral mini 4b realtime quant should you choose?
Q8_0 (4.7 GB) is the middle option in this repository. Close to lossless. It costs roughly twice the memory of a 4-bit file, so it suits cases where you have plenty of VRAM.
If memory is tight, the smallest file is Q4_K at 2.5 GB, with the quality trade-offs that come with fewer bits. The usual sweet spot between file size and quality, and the quant most people start with.
New to quant names? Read GGUF quantization explained.
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about voxtral mini 4b realtime
How much VRAM does voxtral mini 4b realtime need?
About 5.8 GB at Q8_0 with an 8K context window. That is the 4.7 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run voxtral mini 4b realtime on an 8 GB GPU?
Yes, with Q8_0: the 4.7 GB file needs about 5.8 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of voxtral mini 4b realtime GGUF?
It depends on the quant: Q4_K is 2.5 GB, Q8_0 is 4.7 GB, F16 is 8.9 GB.
Where can I download voxtral mini 4b realtime GGUF?
The files are on Hugging Face in the cstr/voxtral-mini-4b-realtime-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does voxtral mini 4b realtime use?
Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run Voxtral Mini 4B Realtime on 8 GB VRAM?
Yes, using the Q4_K quantization which requires about 3.3 GB of VRAM at 8k context.
Does the model support vision tasks?
No, the model is designed only for audio transcription.
What is the default transcription delay?
The default delay is 480 ms, adjustable in multiples of 80 ms.