LLaVA GGUF models

77 GGUF models in the LLaVA family, quantized by 44 uploaders. Sizes run from 374M to 34.4B parameters. The longest trained context is 512K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
LL
ggml llava v1.5 7b
LLaVA · mys
Params6.74B
Context4K
Smallest4.1 GB
Vision
LL
llava 1.6
LLaVA · cmp-nct
Params7.24B
Context32K
Smallest4.4 GB
Vision
LL
LLaVA-NeXT-Video-7B-DPO
LLaVA · ManishThota
Params6.74B
Context4K
Smallest7.2 GB
EmbeddingVision
LL
Llava v1.6 Vicuna 7B
LLaVA · second-state
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
Llava v1.5 7B
LLaVA · second-state
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
nanoLLaVA
LLaVA · abetlen
Params620M
Context32K
Smallest1.2 GB
Vision
LL
llava v1.6 34B
LLaVA · cjpais
Params34.4B
Context4K
Q4_K_M20.7 GB
Vision
LL
llava v1.6 vicuna 13b
LLaVA · cjpais
Params13B
Context4K
Q4_K_M7.9 GB
Vision
LL
liuhaotian llava v1.5 13b
LLaVA · PsiPi
Params13B
Context4K
Smallest5.4 GB
Vision
LL
llava v1.6 vicuna 7b
LLaVA · cjpais
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
SpaceLLaVA
LLaVA · remyxai
Params13B
Context4K
Smallest7.4 GB
ReasoningVision
LL
llava
LLaVA · calcuis
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
LLaVA Critic R1 7B Plus Mimo
LLaVA · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
ReasoningVision
LL
nnethercott llava v1.5 7b hf vicuna
LLaVA · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
ggml bakllava 1
LLaVA · mys
Params7.24B
Context32K
Smallest4.4 GB
Vision
LL
AtomThink LLaVA1.5 7B
LLaVA · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
llava v1.5 7b gpt4OCR hf GGUF Q4 K M
LLaVA · nnethercott
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
Llava 1.5 13b Hf
LLaVA · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
Vision
LL
_Spydaz_Web_AI_Llava
LLaVA · RichardErkhov
Params7.24B
Context512K
Q4_K_M4.4 GB
Vision
LL
llava v1.5 7b gpt4OCR hf GGUF Q2 K
LLaVA · nnethercott
Params6.74B
Context4K
Smallest2.5 GB
Vision
LL
ggml llava v1.5 13b
LLaVA · mys
Params13B
Context4K
Smallest7.9 GB
Vision
LL
AtomThink LLaVA1.5 7B
LLaVA · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
BakLLaVA 1
LLaVA · abetlen
Params7.24B
Context32K
Smallest3.8 GB
Vision
LL
llava v1.5 7b hf vicuna
LLaVA · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
Llava 1.5 13b Hf
LLaVA · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
Vision
LL
Turkish LLaVA v0.1 Q4 K M
LLaVA · atasoglu
Params8.03B
Context8K
Q4_K_M4.9 GB
Vision
LL
LlavaGuard v1.0 13B Hf
LLaVA · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
Vision
LL
Llava 1.1B 20B
LLaVA · mradermacher
Params778M
Context2K
Q4_K_M0.5 GB
Vision
LL
CIM LLaVA1.5 7B
LLaVA · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
Llava v1.5 7b
LLaVA · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision

About the LLaVA GGUF models

LLaVA is represented on ggufmodels.com by 77 GGUF models, quantized by 44 uploaders on Hugging Face. The most downloaded are ggml llava v1.5 7b, llava 1.6 and LLaVA-NeXT-Video-7B-DPO. Parameter counts run from 374M to 34.4B, and the longest trained context window is 512K tokens.

A Q4_K_M file of a LLaVA model is between 0.2 to 20.7 GB, which is the quant most people start with. Most files come from mradermacher (16), MasakiK (4) and mys (3). The quants you will meet most often are Q4_K_M (37), Q5_K_M (32), Q8_0 (31) and F16 (31), where the number is how many models offer that quant.

Frequently asked questions

Which LLaVA GGUF should I download?

The most downloaded LLaVA GGUF here is ggml llava v1.5 7b. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a LLaVA model need?

Q4_K_M files of LLaVA models on this site range from 0.2 to 20.7 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do LLaVA GGUF files come from?

The most common quantizers here are mradermacher, MasakiK and mys. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing