LL

llava v1.5 7b llamafile

LlamaGGUF by mozilla-aiUpdated 2025-04-01
Download on Hugging Face
Parameters6.74B
Context4K
Layers32
Architecturellama
Smallest quant4.1 GB
Quick answer

llava v1.5 7b llamafile needs about 12.0 GB of memory at Q8_0 (a 7.2 GB file) with an 8K context window, so it fits on a 12 GB GPU. This repository has 3 quants, from 4.1 GB (Q4_K) to 13.5 GB (F16).

LLaVA v1.5 7B LlamaFile – multimodal chatbot with vision support

LLaVA v1.5 7B is a 7‑billion‑parameter multimodal model based on Llama. It supports image input and text generation. The model is available in several GGUF quantizations for llama.cpp.

Good at

  • Vision‑enabled text generation
  • Multimodal instruction following
  • Research‑grade LMM capabilities

Limits

  • No vision support in quantized Q4_K?
  • Requires 8‑k context for memory estimates
  • License is llama2 community

Best for

  • Research on multimodal chatbots
  • Hobbyist experimentation with vision models
  • Educational demos of image‑text interaction

Who it’s for. Researchers and hobbyists working in computer vision, NLP, and AI who need a multimodal LLM. The model is suited for academic projects and prototype development.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
Q4_K4.1 GB 6.7 GB
Fits Get file →
Q8_07.2 GB 9.8 GB
Too big Get file →
F1613.5 GB 16.1 GB
Too big Get file →

For 8 GB VRAM, use Q4_K (4.08 GB file, ~8.9 GB memory at 8k context). For 12 GB VRAM, Q8_0 (7.16 GB file, ~12 GB memory). For 18 GB VRAM, F16 (13.48 GB file, ~18.3 GB memory). Estimates are approximate.

What GPU do you need for llava v1.5 7b llamafile?

These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: no quant fits at an 8K context.
  • 6 GB VRAM: no quant fits at an 8K context.
  • 8 GB VRAM: no quant fits at an 8K context.
  • 12 GB VRAM: the best quant that fits is Q8_0 (7.2 GB file, about 12.0 GB in use).
  • 16 GB VRAM: the best quant that fits is Q8_0 (7.2 GB file, about 12.0 GB in use).
  • 24 GB VRAM: the best quant that fits is F16 (13.5 GB file, about 18.3 GB in use).

Which llava v1.5 7b llamafile quant should you choose?

Q8_0 (7.2 GB) is the middle option in this repository. Close to lossless. It costs roughly twice the memory of a 4-bit file, so it suits cases where you have plenty of VRAM.

If memory is tight, the smallest file is Q4_K at 4.1 GB, with the quality trade-offs that come with fewer bits. The usual sweet spot between file size and quality, and the quant most people start with.

New to quant names? Read GGUF quantization explained.

Specifications

Architecture from the GGUF header

Architecturellama
Parameters6.74B
Layers32
Trained context4,096 tokens
KV cache (fp16)512 KB per token

Source

Quantized bymozilla-ai
FamilyLlama
Licensellama2
Downloads (HF)4,124
Last updated2025-04-01

Frequently asked questions about llava v1.5 7b llamafile

How much VRAM does llava v1.5 7b llamafile need?

About 12.0 GB at Q8_0 with an 8K context window. That is the 7.2 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run llava v1.5 7b llamafile on an 8 GB GPU?

Not at an 8K context. The smallest file needs about 8.9 GB, so you would need a larger GPU or CPU offloading.

What is the file size of llava v1.5 7b llamafile GGUF?

It depends on the quant: Q4_K is 4.1 GB, Q8_0 is 7.2 GB, F16 is 13.5 GB.

What context length does llava v1.5 7b llamafile support?

The GGUF header lists a trained context length of 4,096 tokens (4K). The KV cache grows with the context you use: about 4.3 GB at 8K and 2.1 GB at 4K.

Where can I download llava v1.5 7b llamafile GGUF?

The files are on Hugging Face in the mozilla-ai/llava-v1.5-7b-llamafile repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does llava v1.5 7b llamafile use?

Hugging Face lists the license as llama2. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run LLaVA v1.5 7B on 8 GB VRAM?

Yes, using the Q4_K quantization which requires about 8.9 GB of VRAM at 8k context.

Does the model support vision?

Yes, it can process images and generate text responses.

What license applies to this model?

The model is licensed under the Llama 2 Community License.

Similar models

Browse all →

More from mozilla-ai

All by mozilla-ai →

More Llama models

All Llama →

Other small models

All small models →