GE

medical gemma 3n 4b

GemmaGGUF by ericriscoUpdated 2025-07-28
Download on Hugging Face
Parameters6.87B
Context32K
Layers35
Architecturegemma3n
Smallest quant13.7 GB
Quick answer

medical gemma 3n 4b needs about 14.8 GB of memory at its main file (a 13.7 GB file) with an 8K context window, so it fits on a 16 GB GPU. Longer context needs more: at 32K tokens the KV cache alone adds about 2.3 GB.

Medical Gemma 3n 4B – lightweight medical LLM in GGUF format

Medical Gemma 3n 4B is a 6.87B parameter model from the Gemma family, fine‑tuned for medical text. It is distributed as a single GGUF file and supports 32K context tokens. The model is text‑only and runs offline via Ollama.

Good at

  • Medical text generation
  • Instruction‑tuned responses
  • Offline inference on consumer hardware

Limits

  • Only text input supported
  • No vision or multimodal capabilities
  • Requires 14.8GB VRAM at 8K context

Best for

  • Medical chat assistants
  • Clinical note drafting
  • Emergency triage support

Who it’s for. Developers building local medical assistants who need a small, instruction‑tuned model that can run on consumer GPUs. The model is ideal for offline use cases where privacy and speed are critical.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
—13.7 GB 14.8 GB
Too big Get file →

The provided GGUF file is 13.74GB and estimates 14.8GB VRAM at 8K context. It is suitable for GPUs with 16GB or more VRAM. No other quantizations are listed, so use this file for best performance.

What GPU do you need for medical gemma 3n 4b?

These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: no quant fits at an 8K context.
  • 6 GB VRAM: no quant fits at an 8K context.
  • 8 GB VRAM: no quant fits at an 8K context.
  • 12 GB VRAM: no quant fits at an 8K context.
  • 16 GB VRAM: the best quant that fits is the main file (13.7 GB file, about 14.8 GB in use).

Specifications

Architecture from the GGUF header

Architecturegemma3n
Parameters6.87B
Layers35
Trained context32,768 tokens
KV cache (fp16)70 KB per token

Source

Quantized byericrisco
FamilyGemma
Licensegemma
Downloads (HF)1,122
Last updated2025-07-28

Frequently asked questions about medical gemma 3n 4b

How much VRAM does medical gemma 3n 4b need?

About 14.8 GB at its main file with an 8K context window. That is the 13.7 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run medical gemma 3n 4b on an 8 GB GPU?

Not at an 8K context. The smallest file needs about 14.8 GB, so you would need a larger GPU or CPU offloading.

What is the file size of medical gemma 3n 4b GGUF?

It depends on the quant: file is 13.7 GB.

What context length does medical gemma 3n 4b support?

The GGUF header lists a trained context length of 32,768 tokens (32K). The KV cache grows with the context you use: about 0.6 GB at 8K and 2.3 GB at 32K.

Where can I download medical gemma 3n 4b GGUF?

The files are on Hugging Face in the ericrisco/medical-gemma-3n-4b-gguf repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does medical gemma 3n 4b use?

Hugging Face lists the license as gemma. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run Medical Gemma 3n 4B on 8GB VRAM?

The model requires about 14.8GB VRAM at 8K context, so 8GB VRAM is insufficient.

Does this model support image input?

No, only text input is supported.

What is the context length of this model?

The model supports up to 32K tokens of context.

Other GGUFs of gemma-3n-E4B-it

Similar models

Browse all →

More from ericrisco

All by ericrisco →

More Gemma models

All Gemma →

Other small models

All small models →