GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
SH
shisa v2.1c lfm2 350m
Shisa · mradermacher
Params354M
Context125K
Best fitF16 1.7 GB
GGUF
UN
Unity Coder 7b
Unity · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
CodingTool use
MI
Minitron 4B Base
Minitron · legraphista
Params4.19B
Context4K
Best fitQ3_K_L 4.0 GB
GGUF
NE
Nemotron 3 Embed 8B BF16
Nemotron · Aqua00
Params7.95B
Context256K
Best fitIQ2_XXS 4.0 GB
Embedding
JI
Jina Reranker M0
Jina · jinaai
Params1.54B
Context32K
Best fitF16 3.8 GB
Embedding
BL
Blossom V6.2 8B
Blossom · mradermacher
Params8.19B
Context32K
Best fitIQ1_M 4.0 GB
GGUF
MA
Mamba 370m Hf
Mamba · mradermacher
Params372M
Context1024K
Best fitF16 1.3 GB
GGUF
LL
Dolphin 3.0 Llama 3.2 3B
Llama · QuantFactory
Params3.61B
Context128K
Best fitQ5_0 4.0 GB
CodingTool use
LF
Kulyk Uk En
Lfm · Yehor
Params354M
Context125K
Best fitQ8_0 1.4 GB
GGUF
QW
Qwen2.5 VL 3B Instruct
Qwen · DhruvalLabs
Params3.09B
Context125K
Best fitQ6_K 3.3 GB
Vision
QW
LLDS A GRPO Qwen2.5 7B Ins i1
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
AN
ANTRU 3B
Antru · mradermacher
Params3.09B
Context32K
Best fitQ6_K 3.3 GB
Tool use
MI
Huihui Devstral Small 2 24B Instruct 2512 Abliterated
Mistral · mradermacher
Params23.6B
Context384K
Q4_K_M14.3 GB
Coding
LL
Llama 3.1 8B Instruct RoleVerse SFT DPO
Llama · mradermacher
Params8.03B
Context128K
Best fitIQ2_XXS 4.0 GB
RoleplayTool use
QW
Qwen3Guard Gen 0.6B
Qwen · mradermacher
Params752M
Context32K
Best fitF16 2.9 GB
GGUF
QW
Qwen3 Sex
Qwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
GGUF
GE
Gemma Kimu 9b It
Gemma · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
BA
T5 Base
Base · chatpig
Params223M
Context512
Best fitF32 1.2 GB
Coding
QW
Qwen3 Yoyo V4 42B A3B Thinking TOTAL RECALL PKDick V
Qwen · mradermacher
Params42.4B
Context256K
Q4_K_M25.7 GB
CodingReasoning
QW
Qwen3.5 9B DS9 USS Defiant
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Roleplay
LL
Llama3.1 8B PlumMath
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
TI
TIPO 100M
Tipo · KBlueLeaf
Params100M
Context8K
Best fitF16 0.9 GB
Vision
LL
Discoleo
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
QW
Qwen3 Esper3 Reasoning Instruct 6B Brainstorm20x Enhanced E32 128k Ctx
Qwen · mradermacher
Params5.94B
Context128K
Best fitIQ1_M 3.9 GB
CodingReasoning
LA
Laguna S 2.1
Laguna · bartowski
Params118B
Context1024K
Q4_K_M71.8 GB
GGUF
LL
Meta-Llama-3-8B-Instruct
Llama · bartowski
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
QW
Qwen3.8-27B
Qwen · SeraphicHero
Params27.3B
Context256K
Q4_K_M16.5 GB
Vision
GR
granite-4.0-h-350m
Granite · Mungert
Params340M
Context1024K
Best fitBF16 2.0 GB
GGUF
PY
pythia-1.4b-dpo-full
Pythia · mradermacher
Params1.41B
Context2K
Best fitQ8_0 3.6 GB
GGUF
QW
Samantha-qwen-0.5b-ptbr
Qwen · mradermacher
Params494M
Context32K
Best fitQ6_K 1.1 GB
GGUF

Other memory sizes

Keep browsing