GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
SM
SmolLM2 135M
SmolLM · prithivMLmods
Params135M
Context8K
Best fitF16 1.0 GB
PH
Phi 4 Mini Instruct
Phi · lmstudio-community
Params3.84B
Context128K
Best fitQ3_K_L 3.8 GB
TE
Ternary Bonsai 4B
Ternary · prism-ml
Params4B
Context32K
Smallest1.1 GB
OC
Occamy 1.0 APEX I MiniPlus V2.1
Occamy · IsValorum
Params34.7B
Context256K
Best fitQ8_0 3.2 GB
IN
InternVL3_5 2B
Internvl · bartowski
Params2.03B
Context40K
Best fitQ8_0 3.6 GB
LL
LIama 3.2 test
Llama · bhanuripranathi
Params3.21B
Context128K
Best fitQ4_K_M 3.5 GB
QW
Qwen3.8 Flash Next
Qwen · AesSedai
Params180B
Context256K
Q4_K_M116.1 GB
AR
AREX Turbo
Arex · bartowski
Params4.2B
Context256K
Best fitQ3_K_M 4.0 GB
QW
Huihui Qwen3.6 35B A3B Claude 4.7 Opus Abliterated
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen2.5 Coder 1.5B
Qwen · ggml-org
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
HY
Hy MT2 1.8B 1.25Bit
tencent
Params1.79B
Context256K
Smallest0.5 GB
JE
Jebadiah 9b v2
Jebadiah · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
TI
TinyLlama 1.1B Chat v0.6
Tinyllama · TinyLlama
Params1.1B
Context2K
Best fitQ4_0 1.3 GB
GE
Gemma 3 270m Qat Q4_0 Unquantized
Gemma · ggml-org
Params268M
Context32K
Best fitQ4_0 0.9 GB
QW
Kataguru Sceptic 35B A3B v2.6
Qwen · kataguru
Params35.2B
Context256K
Best fitQ8_0 3.2 GB
QW
Qwen2.5-Omni-3B
Qwen · unsloth
Params3.4B
Context32K
Best fitQ6_K_XL 3.9 GB
NE
Nemotron 3.5 Bolt E4B GSQ RCO
Nemotron · abenzerps
Params7.52B
Context128K
Best fitBF16 1.4 GB
IN
InternLM2
InternLM · legraphista
Params7.74B
Context8K
Best fitIQ2_XS 4.0 GB
NE
NVIDIA Nemotron 3.5 Bolt E4B Pollard
Nemotron · PollardWeights
Params7.52B
Context128K
Best fitIQ2_XXS 3.8 GB
QW
Qwen2.5 3B Instruct
Qwen · lmstudio-community
Params3.09B
Context32K
Best fitQ6_K 3.3 GB
QW
Huihui Qwen3 4B Abliterated v2
Qwen · mradermacher
Params4B
Context40K
Best fitIQ4_XS 4.0 GB
QW
Qwen3 4B Thinking 2507
Qwen · bartowski
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
DE
DeepSeek V4.1 Flash 1017 ELVEZ Q2
DeepSeek · elvezjp
Params485M
Context—
Smallest1.0 GB
PH
phonon 40m fr
Phonon · gradium
Params51M
Context—
Smallest0.1 GB
QW
Qwen2.5 VL 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
QW
Qwen3.5 0.8 Cyber
Qwen · reaperdoesntknow
Params752M
Context256K
Best fitBF16 2.4 GB
DE
DeepSeek R1 Distill Qwen 7B Uncensored
DeepSeek · mradermacher
Params7.61B
Context128K
Best fitQ2_K 4.0 GB
MA
MangaLMM
Mangalmm · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
QW
Qwen3.5 2B CyberSec
Qwen · reaperdoesntknow
Params1.94B
Context256K
Best fitQ8_0 3.0 GB
ZA
Zawen v0.1 35B A3B
Zawen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB