GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
MI
Ministral 3 14B Instruct 2512
Mistral · bartowski
Params13.5B
Context256K
Q4_K_M8.2 GB
GR
Granite-4.0-350M
Granite · unsloth
Params352M
Context32K
Best fitBF16 2.1 GB
NA
Nanbeige4.2 3B
Nanbeige · owao
Params4.17B
Context256K
Best fitQ4_K_M 3.8 GB
GL
GLM 4.5 Air Derestricted
GLM · bartowski
Params110B
Context128K
Q4_K_M73.5 GB
QW
Qwen2.5 Coder 3B Instruct
Qwen · lmstudio-community
Params3.09B
Context32K
Best fitQ6_K 3.3 GB
QW
Qwen3.8 27B Optimal Ablit Heretic
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
EU
EuroLLM 1.7B Instruct
Eurollm · mradermacher
Params1.66B
Context8K
Best fitQ8_0 3.1 GB
QW
Qwen3 30B A3B Thinking 2507
Qwen · bartowski
Params30.5B
Context256K
Q4_K_M18.6 GB
QW
Qwen3 4b Heretic
Qwen · DreamFast
Params11.8B
Context40K
Best fitQ3_K_M 3.8 GB
FA
Fara1.5 27B
Fara · bartowski
Params27B
Context256K
Q4_K_M17.5 GB
QW
Huihui Qwen3.5 27B Abliterated
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GP
Gpt Oss 20b Heretic
gpt-oss · bartowski
Params20.9B
Context128K
Q4_K_M15.9 GB
QW
Qwen3.8 27B Antislop
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
QW
Qwen3.8-Flash-Next
Qwen · ji-farthing
Params177B
Context256K
Smallest2.4 GB
GP
Gpt Oss 120b Abliterated
gpt-oss · mradermacher
Params117B
Context128K
Q4_K_M87.9 GB
SU
SuperQwen3.8-27b-abliterated
Superqwen · Jiunsong
Params27B
Context256K
Q4_K_M16.5 GB
DE
deepseek-ai_deepseek-coder-1.3b-base
DeepSeek · TheBloke
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
LL
Dolphin 3.0 Llama 3.2 1b
Llama · bartowski
Params1.24B
Context128K
Best fitF16 3.2 GB
LF
LFM2.5 24B A2B
Lfm · mradermacher
Params24B
Context128K
Q4_K_M14.5 GB
QW
Qwen3.5 122B A10B Abliterated v1
Qwen · mradermacher
Params122B
Context256K
Q4_K_M74.2 GB
QW
Qwen2.5 Coder 32B Instruct 3MPER0RR Abliterated
Qwen · mradermacher
Params32.8B
Context32K
Q4_K_M19.9 GB
MA
UI Mate 27B
Mate · bartowski
Params27B
Context256K
Q4_K_M17.5 GB
HE
HERO 9B
Hero · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
SM
SmolVLM2-256M-Video-Instruct
Smolvlm · ggml-org
Params163M
Context8K
Best fitBF16 1.0 GB
QW
Qwen2.5 1.5b
Qwen · ElementMerc
Params1.54B
Context32K
Best fitF16 3.8 GB
LF
LFM2.5 24B A2B
Lfm · bartowski
Params24B
Context128K
Q4_K_M15.4 GB
EU
EuroLLM 22B Instruct 2512
Eurollm · bartowski
Params22.6B
Context32K
Q4_K_M13.7 GB
FI
Finch 7b
Finch · latestissue
Params7.64B
Context1024K
Best fitQ2_K 3.8 GB
MU
Muse-Glimmer-30B
Muse · ggml-org
Params27.9B
Context128K
Best fitQ8_0 3.7 GB
QW
Qwen3.8 27B Uncensored
Qwen · MiawTeam
Params27.3B
Context256K
Best fitF16 3.6 GB