MA

Marvis MLocalModel3.6.2 35B Q4 K M

MarvisGGUF by gatilinUpdated 2026-07-21
Download on Hugging Face
Parameters26.6B
Context256K
Layers40
Architectureqwen35moe
Smallest quant16.3 GB
Quick answer

Marvis MLocalModel3.6.2 35B Q4 K M needs about 17.4 GB of memory at its main file (a 16.3 GB file) with an 8K context window, so it fits on a 24 GB GPU. Longer context needs more: at 32K tokens the KV cache alone adds about 2.7 GB.

Marvis MLocalModel3.6.2 35B Q4_K_M – 26.6B parameter local LLM

Marvis MLocalModel3.6.2 35B Q4_K_M is a 26.6B parameter Qwen35moe architecture model. It supports reasoning, tools, and vision. The GGUF file is 16.27 GB and requires about 17.4 GB of VRAM at 8 k context.

Good at

  • Reasoning and complex queries
  • Tool usage integration
  • Vision multimodal tasks

Limits

  • Requires 8 k context or less
  • No explicit license listed
  • No native vision support in the main model

Best for

  • Local inference on high‑VRAM GPUs
  • Fine‑tuning with llama.cpp
  • Developing multimodal applications

Who it’s for. Users who need a large, local LLM with reasoning, tool, and vision capabilities. Suitable for developers and researchers working on multimodal AI projects.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
—16.3 GB 17.4 GB
Too big Get file →

The only available quant is Q4_K_M. The 16.27 GB GGUF file needs roughly 17.4 GB of VRAM at 8 k context. For 8 GB VRAM, the model is not feasible. For 16 GB VRAM, it is borderline; consider reducing context size.

What GPU do you need for Marvis MLocalModel3.6.2 35B Q4 K M?

These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: no quant fits at an 8K context.
  • 6 GB VRAM: no quant fits at an 8K context.
  • 8 GB VRAM: no quant fits at an 8K context.
  • 12 GB VRAM: no quant fits at an 8K context.
  • 16 GB VRAM: no quant fits at an 8K context.
  • 24 GB VRAM: the best quant that fits is the main file (16.3 GB file, about 17.4 GB in use).

Specifications

Architecture from the GGUF header

Architectureqwen35moe
Parameters26.6B (approximate, mixture of experts)
Layers40
Trained context262,144 tokens
KV cache (fp16)80 KB per token

Source

Quantized bygatilin
FamilyMarvis
Downloads (HF)1,015
Last updated2026-07-21

Frequently asked questions about Marvis MLocalModel3.6.2 35B Q4 K M

How much VRAM does Marvis MLocalModel3.6.2 35B Q4 K M need?

About 17.4 GB at its main file with an 8K context window. That is the 16.3 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run Marvis MLocalModel3.6.2 35B Q4 K M on an 8 GB GPU?

Not at an 8K context. The smallest file needs about 17.4 GB, so you would need a larger GPU or CPU offloading.

What is the file size of Marvis MLocalModel3.6.2 35B Q4 K M GGUF?

It depends on the quant: file is 16.3 GB.

What context length does Marvis MLocalModel3.6.2 35B Q4 K M support?

The GGUF header lists a trained context length of 262,144 tokens (256K). The KV cache grows with the context you use: about 0.7 GB at 8K and 2.7 GB at 32K.

Where can I download Marvis MLocalModel3.6.2 35B Q4 K M GGUF?

The files are on Hugging Face in the gatilin/Marvis-MLocalModel3.6.2-35B-Q4_K_M repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

Can I run Marvis MLocalModel3.6.2 on 8 GB VRAM?

No. The model requires about 17.4 GB of VRAM at 8 k context, exceeding 8 GB.

Does the model support vision tasks?

Yes, it includes a vision multimodal projection weight of about 861 MB for vision tasks.

What engine is needed to run this model?

Use llama.cpp-metal, version b9846, as specified in the repo card.

Similar models

Browse all →

More from gatilin

All by gatilin →

Other medium models

All medium models →