GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
MI
Mistral Small 24B Instruct 2501 Heretic
Mistral · mradermacher
Params23.6B
Context32K
Q4_K_M14.3 GB
LI
Lil Beastie 4b
Lil · mradermacher
Params4.02B
Context256K
Best fitIQ4_XS 4.0 GB
NE
Neuron 4B Instruct
Neuron · mradermacher
Params4.02B
Context256K
Best fitIQ4_XS 4.0 GB
KI
RP King 12b
King · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
BO
Borealis 270m
Borealis · mradermacher
Params268M
Context32K
Best fitIQ1_S 0.9 GB
GL
zai org GLM 5.1
GLM · bartowski
Params754B
Context198K
Q4_K_M459.7 GB
G9
G9v3 3B
NANI-Nithin
Params3B
Context128K
Best fitQ6_K 3.4 GB
HU
Hunyuan 4B Instruct
Hunyuan · bartowski
Params4.22B
Context256K
Best fitQ3_K_M 3.9 GB
GR
Granite 4.1 8b Abliterated AND Disinhibited
Granite · mradermacher
Params8.38B
Context128K
Best fitIQ1_M 4.0 GB
GL
Sentie1.0 3B Claude Fable 5 GPT5.2 Sol Kimi K3 GLM 5.2
GLM · SentieAI
Params3.93B
Context256K
Best fitQ5_K_M 3.9 GB
QW
QiMing Holos Plus Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
QW
Qwen2.5 3B Instruct Heretic
Qwen · mradermacher
Params3.09B
Context32K
Best fitQ6_K 3.3 GB
LI
LightOnOCR 3 4B
Lightonocr · richardyoung
Params4.2B
Context256K
Best fitIQ3_M 3.7 GB
SA
Salience 1.5 Flash
Salience · bartowski
Params30.5B
Context256K
Q4_K_M18.7 GB
QW
Qwen3 VL 30B A3B Thinking
Qwen · bartowski
Params30.5B
Context256K
Q4_K_M18.6 GB
LL
Mini Lama 8B Base
Llama · mradermacher
Params8.49B
Context256K
Best fitIQ1_M 3.9 GB
SP
Spark X2.6 4B Base
Spark · mradermacher
Params4.11B
Context1024K
Best fitQ3_K_M 3.9 GB
SE
SearchQwen2.5 7B
Searchqwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
PL
Pluto i1
Pluto · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GL
cerebras/GLM-4.5-Air-REAP-82B-A12B
GLM · bartowski
Params85B
Context128K
Q4_K_M56.6 GB
EG
Ege 8b 1.1
Ege · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
CY
Cyberdonia 24B v8
Cyberdonia · bartowski
Params23.6B
Context128K
Q4_K_M14.6 GB
GR
GRM 3.2 Cliff
Grm · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
NA
Nanbeige4.1-3B
Nanbeige · Mungert
Params3.93B
Context256K
Best fitQ5_K_S 4.0 GB
QW
Qwen3.8 Queen 27B
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
QW
Replete LLM V2.5 Qwen 3b
Qwen · bartowski
Params3.4B
Context32K
Best fitQ6_K_L 3.7 GB
KE
Keural Cortex 8B v1.0 1
Keural · mradermacher
Params8.19B
Context64K
Best fitIQ1_M 4.0 GB
QW
Huihui Qwen3 Coder Next Opus 4.6 Reasoning Distilled Abliterated
Qwen · mradermacher
Params79.7B
Context256K
Q4_K_M48.5 GB
QW
Qwen3 Nemotron 32B RLBFF
Qwen · bartowski
Params32.8B
Context40K
Q4_K_M19.8 GB
ED
EdgeRunner Command Nested
Edgerunner · mradermacher
Params7.6B
Context32K
Best fitQ2_K 4.0 GB