GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Meta llama_Meta Llama 3.1 8B Instruct
Llama · ThomasBaruzier
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
GE
Gemma 4 E4B Luchador
Gemma · bartowski
Params7.5B
Context128K
Q4_K_M5.4 GB
Roleplay
LF
LFM2 350M ENJP MT
Lfm · LiquidAI
Params354M
Context125K
Best fitF32 2.5 GB
GGUF
SE
Seed Coder 8B Instruct
Seed · mradermacher
Params8.25B
Context32K
Best fitIQ1_M 3.8 GB
Coding
TI
py007_tinyllama-1.1b-python-v0.1
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
Coding
QW
Qwen2.5 1.5B Instruct
Qwen · mradermacher
Params1.54B
Context32K
Best fitQ6_K 2.0 GB
GGUF
GE
NVIDIA GEM-X SOMA regression denoiser
Gem · LocalAI-io
Params44M
Context120
Best fitF32 1.1 GB
GGUF
MI
Mistral Small 3.1 DRAFT 0.5B
Mistral · bartowski
Params593M
Context32K
Best fitBF16 1.8 GB
GGUF
QW
Qwen2-0.5B-Instruct
Qwen · legraphista
Params494M
Context32K
Best fitBF16 1.6 GB
GGUF
YI
EVA Yi 1.5 9B 32K V1 i1
Yi · mradermacher
Params8.83B
Context32K
Best fitIQ2_XS 4.0 GB
GGUF
GR
Granite-4.0-H-1B
Granite · unsloth
Params1.46B
Context1024K
Best fitQ6_K_XL 3.8 GB
GGUF
MI
Ministral 3 14B Reasoning 2512
Mistral · bartowski
Params13.5B
Context256K
Q4_K_M8.2 GB
ReasoningVision
GP
Gpt2 Medium
Gpt · mradermacher
Params406M
Context1K
Best fitF16 2.1 GB
GGUF
GE
google gemma 2b it
Gemma · RichardErkhov
Params2.51B
Context8K
Best fitQ6_K 2.7 GB
GGUF
PL
Plamo 3 Nict 2b Base
Plamo · mmnga-o
Params2.6B
Context4K
Best fitQ8_0 3.7 GB
GGUF
GP
GPT2-774M-CINDER-SHOW-MULTI-CHAT
Gpt · RichardErkhov
Params838M
Context1K
Best fitQ8_0 2.9 GB
GGUF
QW
Qwen3-Vl-2B-Instruct-1M
Qwen · unsloth
Params1.72B
Context977K
Best fitQ8_K_XL 3.8 GB
Vision
DE
DeepCoder 1.5B Preview
Deepcoder · bartowski
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
Coding
HE
Hermes 4 14B
Hermes · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
ReasoningRoleplayTool use
MA
Maincoder 1B
Maincoder · mradermacher
Params1.03B
Context2K
Best fitF16 3.0 GB
Coding
QW
Qwen3.8 27B DFlash2
Qwen · Anbeeld
Params1.92B
Context256K
Best fitQ8_0 2.7 GB
GGUF
TI
py007_tinyllama-1.1b-intermediate-step-480k-1t
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
Coding
QW
Qwen3.5 4B MultiDistill 3xMerge UNCENSORED Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
ReasoningVision
OX
OxCoder 9B
Oxcoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Coding
QW
Qwen2.5 Coder 1.5B Shellsmith
Qwen · ajayk007
Params1.54B
Context32K
Best fitF16 3.8 GB
Coding
NO
nomic-embed-code
Nomic · nomic-ai
Params7.1B
Context32K
Best fitQ2_K 3.8 GB
CodingEmbedding
US
UserLM 8b
Userlm · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LF
Lfm2.5-1.2B-Thinking
Lfm · unsloth
Params1.17B
Context125K
Best fitBF16 3.9 GB
Reasoning
CH
MN CharThink 12B Base
Charthink · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
QW
Qwen3.8 Flash Next MTP
Qwen · quimmedes
Params3.88B
Context256K
Best fitQ4_0 3.7 GB
GGUF

Other memory sizes

Keep browsing