MI

Mistral Small 4 119B 2603 GGUF HALO

MistralGGUF by BeinseziiUpdated 2026-03-17
Download on Hugging Face
Parameters119B
Context—
Layers—
Architecture—
Smallest quant0.1 GB
Quick answer

Mistral Small 4 119B 2603 GGUF HALO needs about 0.6 GB of memory at its main file (a 0.1 GB file) with an 8K context window, so it fits on a 4 GB GPU. This repository has 2 quants, from 0.1 GB to 98.4 GB.

Mistral Small 4 119B 2603 GGUF HALO – a compact, multimodal instruction and reasoning model

Mistral Small 4 119B 2603 GGUF HALO is a 119‑billion‑parameter model quantized for efficient inference. It supports text and image input, offers a 256k context window, and is licensed under Apache 2.0 for commercial and non‑commercial use.

Good at

  • Multimodal text and image input
  • Large 256k context window
  • Fast inference with 8‑bit quantization

Limits

  • Requires 110.7 GB VRAM for full 98.39 GB quant
  • No explicit vision support in the quantized file list
  • Limited to text output only in the provided quant

Best for

  • Chatbots needing large context
  • Document summarization with image references
  • AI agents with function calling

Who it’s for. Developers building AI assistants that require both instruction following and reasoning, and who can provision high‑memory GPUs or multi‑GPU setups. The model is suitable for enterprises needing a licensed, open‑source solution.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
—0.1 GB 0.6 GB
Fits Get file →
—98.4 GB 110.7 GB
Too big Get file →

For systems with 8 GB VRAM, use the 0.12 GB quant file; it estimates 0.6 GB VRAM usage at 8k context. For larger GPUs, the 98.39 GB quant provides higher quality but requires approximately 110.7 GB VRAM at 8k context. Memory estimates are approximate.

What GPU do you need for Mistral Small 4 119B 2603 GGUF HALO?

These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: the best quant that fits is the main file (0.1 GB file, about 0.6 GB in use).
  • 6 GB VRAM: the best quant that fits is the main file (0.1 GB file, about 0.6 GB in use).
  • 8 GB VRAM: the best quant that fits is the main file (0.1 GB file, about 0.6 GB in use).
  • 12 GB VRAM: the best quant that fits is the main file (0.1 GB file, about 0.6 GB in use).
  • 16 GB VRAM: the best quant that fits is the main file (0.1 GB file, about 0.6 GB in use).
  • 24 GB VRAM: the best quant that fits is the main file (0.1 GB file, about 0.6 GB in use).
  • 32 GB VRAM: the best quant that fits is the main file (0.1 GB file, about 0.6 GB in use).
  • 48 GB VRAM: the best quant that fits is the main file (0.1 GB file, about 0.6 GB in use).

Specifications

Architecture from the GGUF header

Parameters119B

Source

Quantized byBeinsezii
FamilyMistral
Licenseapache-2.0
Downloads (HF)64
Last updated2026-03-17

Frequently asked questions about Mistral Small 4 119B 2603 GGUF HALO

How much VRAM does Mistral Small 4 119B 2603 GGUF HALO need?

About 0.6 GB at its main file with an 8K context window. That is the 0.1 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run Mistral Small 4 119B 2603 GGUF HALO on an 8 GB GPU?

Yes, with the smaller files: the 0.1 GB file needs about 0.6 GB at 8K context. Larger quants in this repository do not fit in 8 GB.

What is the file size of Mistral Small 4 119B 2603 GGUF HALO GGUF?

It depends on the quant: file is 0.1 GB, file is 98.4 GB.

Where can I download Mistral Small 4 119B 2603 GGUF HALO GGUF?

The files are on Hugging Face in the Beinsezii/Mistral-Small-4-119B-2603-GGUF-HALO repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does Mistral Small 4 119B 2603 GGUF HALO use?

Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run Mistral Small 4 on an 8 GB GPU?

Only the 0.12 GB quant file can be loaded, estimating 0.6 GB VRAM usage at 8k context. The larger 98.39 GB quant needs over 110 GB VRAM.

Does the model support image input?

Yes, the base model accepts multimodal input, but the provided quant files are text‑only; image support is not included in the quantized files.

What license applies to this model?

The model is released under the Apache 2.0 license, allowing commercial and non‑commercial use.

Other GGUFs of Mistral-Small-4-119B-2603

Similar models

Browse all →

More from Beinsezii

All by Beinsezii →

More Mistral models

All Mistral →

Other large models

All large models →