MI

voxtral mini 4b realtime

MistralGGUF by cstrUpdated 2026-09-17
Download on Hugging Face
Parameters4.43B
Context—
Layers—
Architecturevoxtral4b
Smallest quant2.5 GB
Quick answer

voxtral mini 4b realtime needs about 5.8 GB of memory at Q8_0 (a 4.7 GB file) with an 8K context window, so it fits on a 6 GB GPU. This repository has 3 quants, from 2.5 GB (Q4_K) to 8.9 GB (F16).

4.4B‑parameter realtime speech‑to‑text model

Voxtral Mini 4B Realtime is a 4.43‑billion‑parameter model designed for low‑latency audio transcription. It uses a causal audio encoder and a 26‑layer Mistral decoder. The model supports 13 languages and is available under Apache 2.0.

Good at

  • Realtime speech‑to‑text
  • Low‑latency transcription
  • Multilingual support

Limits

  • No vision support
  • Requires vLLM for production use
  • Maximum model length limited by RoPE pre‑allocations

Best for

  • Live podcast transcription
  • Real‑time captioning
  • Audio‑driven chatbots

Who it’s for. Developers building real‑time speech‑to‑text applications who need a lightweight, open‑source model. The model is suitable for servers with moderate GPU memory and for users who prefer a causal encoder for streaming.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
Q4_K2.5 GB 3.3 GB
Fits Get file →
Q8_04.7 GB 5.8 GB
Fits Get file →
F168.9 GB 10.4 GB
Too big Get file →

For 8 GB VRAM use Q4_K (file size 2.52 GB, estimated memory 3.3 GB at 8k context). For 12 GB VRAM Q8_0 (4.73 GB, 5.8 GB). F16 requires 8.88 GB and 10.4 GB memory, suitable for high‑end GPUs.

What GPU do you need for voxtral mini 4b realtime?

These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: the best quant that fits is Q4_K (2.5 GB file, about 3.3 GB in use).
  • 6 GB VRAM: the best quant that fits is Q8_0 (4.7 GB file, about 5.8 GB in use).
  • 8 GB VRAM: the best quant that fits is Q8_0 (4.7 GB file, about 5.8 GB in use).
  • 12 GB VRAM: the best quant that fits is F16 (8.9 GB file, about 10.4 GB in use).

Which voxtral mini 4b realtime quant should you choose?

Q8_0 (4.7 GB) is the middle option in this repository. Close to lossless. It costs roughly twice the memory of a 4-bit file, so it suits cases where you have plenty of VRAM.

If memory is tight, the smallest file is Q4_K at 2.5 GB, with the quality trade-offs that come with fewer bits. The usual sweet spot between file size and quality, and the quant most people start with.

New to quant names? Read GGUF quantization explained.

Specifications

Architecture from the GGUF header

Architecturevoxtral4b
Parameters4.43B

Source

Quantized bycstr
FamilyMistral
Licenseapache-2.0
Downloads (HF)3,807
Last updated2026-09-17

Frequently asked questions about voxtral mini 4b realtime

How much VRAM does voxtral mini 4b realtime need?

About 5.8 GB at Q8_0 with an 8K context window. That is the 4.7 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run voxtral mini 4b realtime on an 8 GB GPU?

Yes, with Q8_0: the 4.7 GB file needs about 5.8 GB at 8K context. Larger quants in this repository do not fit in 8 GB.

What is the file size of voxtral mini 4b realtime GGUF?

It depends on the quant: Q4_K is 2.5 GB, Q8_0 is 4.7 GB, F16 is 8.9 GB.

Where can I download voxtral mini 4b realtime GGUF?

The files are on Hugging Face in the cstr/voxtral-mini-4b-realtime-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does voxtral mini 4b realtime use?

Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run Voxtral Mini 4B Realtime on 8 GB VRAM?

Yes, using the Q4_K quantization which requires about 3.3 GB of VRAM at 8k context.

Does the model support vision tasks?

No, the model is designed only for audio transcription.

What is the default transcription delay?

The default delay is 480 ms, adjustable in multiples of 80 ms.

Other GGUFs of Voxtral-Mini-4B-Realtime-2602

Similar models

Browse all →

More from cstr

All by cstr →

More Mistral models

All Mistral →

Other small models

All small models →