GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
qwen3.5 4B typescript coder
Qwen · MassivDash
Params4.2B
Context256K
Best fitQ3_K_M 3.8 GB
CodingVision
YI
Yi 1.5 6B Chat
Yi · RichardErkhov
Params6.06B
Context4K
Best fitQ3_K 4.0 GB
GGUF
GE
gemma3
Gemma · libretranslate
Params11.8B
Context32K
Best fitQ4_0 3.9 GB
GGUF
TE
Tess 4 27B
Tess · bartowski
Params27.3B
Context256K
Q4_K_M17.8 GB
ReasoningTool useVision
MI
Configurable-Mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
PY
pythia 160m
Pythia · QuantFactory
Params162M
Context2K
Best fitQ8_0 1.0 GB
GGUF
QW
Qwen2.5 7B Comment Sexuality Reasoner
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
Reasoning
BA
BanglaLLama-3-8b-BnWiki-Base
Banglallama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
LL
Llama 3.2 3b NSFW Aesir Uncensored
Llama · Novaciano
Params3.61B
Context128K
Smallest2.0 GB
Roleplay
MI
MiroThinker 1.7
Mirothinker · mradermacher
Params235B
Context256K
Q4_K_M142.2 GB
GGUF
NI
Ninja v1 NSFW 128k
Ninja · mmnga
Params7.24B
Context128K
Best fitIQ2_S 3.9 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
GR
Granite 4.0 Micro
Granite · bartowski
Params3.4B
Context128K
Q4_K_M2.1 GB
GGUF
GE
Gemma 4 E2B it qat NVFP4
Gemma · WhiskyAKM
Params4.63B
Context128K
Smallest0.1 GB
Vision
VE
VertaLily 1.2 1B
Vertalily · VLTX
Params1.17B
Context125K
Best fitQ8_0 2.8 GB
GGUF
QW
Parable Qwen3 8B Claude Fable 5
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
Reasoning
QW
imajev 2b
Qwen · mindchain
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
Vision
LL
Llama 3.2 3B Instruct Abliterated
Llama · RichardErkhov
Params3.61B
Context128K
Best fitQ5_0 4.0 GB
GGUF
GE
Gemma 4 26B A4B StyleTune V2 QAT
Gemma · alexisStacksCode
Params25.2B
Context128K
Best fitQ4_0 1.8 GB
Vision
LF
Lfm2-1.2B
Lfm · unsloth
Params1.17B
Context32K
Best fitF16 3.9 GB
GGUF
OR
Ornith 1.5 9B DFlash
Ornith · abenzerps
Params1.29B
Context256K
Best fitBF16 3.3 GB
Reasoning
IC
darkps ice AI
Ice · bartowski
Params8.2B
Context40K
Q4_K_M5.2 GB
GGUF
LL
nvidia Llama 3 3 Nemotron Super 49B v1 5
Llama · bartowski
Params49.9B
Context128K
Q4_K_M30.2 GB
GGUF
HE
Hebrew_Nemo
Hebrew · mradermacher
Params12.2B
Context128K
Q4_K_M7.5 GB
GGUF
MO
Dreamaiai 27B 03
Moon · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
ReasoningTool useVision
RD
RDT2 VQ
Rdt · mradermacher
Params7.6B
Context125K
Best fitQ2_K 4.0 GB
Vision
DE
Andy 3.5
DeepSeek · Sweaterdog
Params7.62B
Context128K
Best fitQ2_K 3.8 GB
Reasoning
BE
BeeLLM 30P
Beellm · schizophyllume
Params12M
Context4K
Smallest0.0 GB
GGUF
GE
Gemma 3 4b It Uncensored DBL X
Gemma · mradermacher
Params4.26B
Context128K
Best fitIQ3_S 3.9 GB
Roleplay
ED
Eduardo 27b
Eduardo · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF

Other memory sizes

Keep browsing