GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
QW
qwen3.5 4B typescript coder
Qwen · MassivDash
Params4.2B
Context256K
Best fitQ3_K_M 3.8 GB
YI
Yi 1.5 6B Chat
Yi · RichardErkhov
Params6.06B
Context4K
Best fitQ3_K 4.0 GB
GE
gemma3
Gemma · libretranslate
Params11.8B
Context32K
Best fitQ4_0 3.9 GB
TE
Tess 4 27B
Tess · bartowski
Params27.3B
Context256K
Q4_K_M17.8 GB
MI
Configurable-Mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
PY
pythia 160m
Pythia · QuantFactory
Params162M
Context2K
Best fitQ8_0 1.0 GB
QW
Qwen2.5 7B Comment Sexuality Reasoner
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
BA
BanglaLLama-3-8b-BnWiki-Base
Banglallama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
LL
Llama 3.2 3b NSFW Aesir Uncensored
Llama · Novaciano
Params3.61B
Context128K
Smallest2.0 GB
MI
MiroThinker 1.7
Mirothinker · mradermacher
Params235B
Context256K
Q4_K_M142.2 GB
NI
Ninja v1 NSFW 128k
Ninja · mmnga
Params7.24B
Context128K
Best fitIQ2_S 3.9 GB
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GR
Granite 4.0 Micro
Granite · bartowski
Params3.4B
Context128K
Q4_K_M2.1 GB
GE
Gemma 4 E2B it qat NVFP4
Gemma · WhiskyAKM
Params4.63B
Context128K
Smallest0.1 GB
VE
VertaLily 1.2 1B
Vertalily · VLTX
Params1.17B
Context125K
Best fitQ8_0 2.8 GB
QW
Parable Qwen3 8B Claude Fable 5
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
QW
imajev 2b
Qwen · mindchain
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
LL
Llama 3.2 3B Instruct Abliterated
Llama · RichardErkhov
Params3.61B
Context128K
Best fitQ5_0 4.0 GB
GE
Gemma 4 26B A4B StyleTune V2 QAT
Gemma · alexisStacksCode
Params25.2B
Context128K
Best fitQ4_0 1.8 GB
LF
Lfm2-1.2B
Lfm · unsloth
Params1.17B
Context32K
Best fitF16 3.9 GB
OR
Ornith 1.5 9B DFlash
Ornith · abenzerps
Params1.29B
Context256K
Best fitBF16 3.3 GB
IC
darkps ice AI
Ice · bartowski
Params8.2B
Context40K
Q4_K_M5.2 GB
LL
nvidia Llama 3 3 Nemotron Super 49B v1 5
Llama · bartowski
Params49.9B
Context128K
Q4_K_M30.2 GB
HE
Hebrew_Nemo
Hebrew · mradermacher
Params12.2B
Context128K
Q4_K_M7.5 GB
MO
Dreamaiai 27B 03
Moon · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
RD
RDT2 VQ
Rdt · mradermacher
Params7.6B
Context125K
Best fitQ2_K 4.0 GB
DE
Andy 3.5
DeepSeek · Sweaterdog
Params7.62B
Context128K
Best fitQ2_K 3.8 GB
BE
BeeLLM 30P
Beellm · schizophyllume
Params12M
Context4K
Smallest0.0 GB
GE
Gemma 3 4b It Uncensored DBL X
Gemma · mradermacher
Params4.26B
Context128K
Best fitIQ3_S 3.9 GB
ED
Eduardo 27b
Eduardo · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB