GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
LL
Meta llama_Meta Llama 3.1 8B Instruct
Llama · ThomasBaruzier
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GE
Gemma 4 E4B Luchador
Gemma · bartowski
Params7.5B
Context128K
Q4_K_M5.4 GB
LF
LFM2 350M ENJP MT
Lfm · LiquidAI
Params354M
Context125K
Best fitF32 2.5 GB
SE
Seed Coder 8B Instruct
Seed · mradermacher
Params8.25B
Context32K
Best fitIQ1_M 3.8 GB
TI
py007_tinyllama-1.1b-python-v0.1
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
QW
Qwen2.5 1.5B Instruct
Qwen · mradermacher
Params1.54B
Context32K
Best fitQ6_K 2.0 GB
GE
NVIDIA GEM-X SOMA regression denoiser
Gem · LocalAI-io
Params44M
Context120
Best fitF32 1.1 GB
MI
Mistral Small 3.1 DRAFT 0.5B
Mistral · bartowski
Params593M
Context32K
Best fitBF16 1.8 GB
QW
Qwen2-0.5B-Instruct
Qwen · legraphista
Params494M
Context32K
Best fitBF16 1.6 GB
YI
EVA Yi 1.5 9B 32K V1 i1
Yi · mradermacher
Params8.83B
Context32K
Best fitIQ2_XS 4.0 GB
GR
Granite-4.0-H-1B
Granite · unsloth
Params1.46B
Context1024K
Best fitQ6_K_XL 3.8 GB
MI
Ministral 3 14B Reasoning 2512
Mistral · bartowski
Params13.5B
Context256K
Q4_K_M8.2 GB
GP
Gpt2 Medium
Gpt · mradermacher
Params406M
Context1K
Best fitF16 2.1 GB
GE
google gemma 2b it
Gemma · RichardErkhov
Params2.51B
Context8K
Best fitQ6_K 2.7 GB
PL
Plamo 3 Nict 2b Base
Plamo · mmnga-o
Params2.6B
Context4K
Best fitQ8_0 3.7 GB
GP
GPT2-774M-CINDER-SHOW-MULTI-CHAT
Gpt · RichardErkhov
Params838M
Context1K
Best fitQ8_0 2.9 GB
QW
Qwen3-Vl-2B-Instruct-1M
Qwen · unsloth
Params1.72B
Context977K
Best fitQ8_K_XL 3.8 GB
DE
DeepCoder 1.5B Preview
Deepcoder · bartowski
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
HE
Hermes 4 14B
Hermes · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
MA
Maincoder 1B
Maincoder · mradermacher
Params1.03B
Context2K
Best fitF16 3.0 GB
QW
Qwen3.8 27B DFlash2
Qwen · Anbeeld
Params1.92B
Context256K
Best fitQ8_0 2.7 GB
TI
py007_tinyllama-1.1b-intermediate-step-480k-1t
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
QW
Qwen3.5 4B MultiDistill 3xMerge UNCENSORED Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
OX
OxCoder 9B
Oxcoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen2.5 Coder 1.5B Shellsmith
Qwen · ajayk007
Params1.54B
Context32K
Best fitF16 3.8 GB
NO
nomic-embed-code
Nomic · nomic-ai
Params7.1B
Context32K
Best fitQ2_K 3.8 GB
US
UserLM 8b
Userlm · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
LF
Lfm2.5-1.2B-Thinking
Lfm · unsloth
Params1.17B
Context125K
Best fitBF16 3.9 GB
CH
MN CharThink 12B Base
Charthink · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
QW
Qwen3.8 Flash Next MTP
Qwen · quimmedes
Params3.88B
Context256K
Best fitQ4_0 3.7 GB