GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
BE
Behemoth X 123B v2.1
Behemoth · bartowski
Params123B
Context128K
Q4_K_M73.2 GB
RO
Rocinante X 12B v1 Heretic Uncensored
Rocinante · mradermacher
Params12.2B
Context128K
Q4_K_M7.5 GB
GR
granite docling 258M
Granite · ggml-org
Params164M
Context8K
Best fitF16 1.0 GB
AV
Average_Normie_v3.69_8B
Average · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
IN
InternVL3 2B Instruct
Internvl · ggml-org
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
NU
NuMarkdown-8B-Thinking
Numarkdown · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
QW
Qwen3.5 9B DFlash
Qwen · Anbeeld
Params1.29B
Context256K
Best fitBF16 3.3 GB
QW
Qwen2.5 7B Instruct
Qwen · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
WR
Wraith 8b
Wraith · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
QW
Qwen3.5 2B GPT 5.1 HighIQ INSTRUCT
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
MU
Muslim 6B v3
Muslim · mradermacher
Params5.94B
Context256K
Best fitIQ1_M 3.9 GB
GE
Gemma 4 E4B It GMCE
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
NE
Next2.5
Next · thelamapi
Params4.2B
Context256K
Best fitIQ4_XS 4.0 GB
QW
Qwen3 5 2b
Qwen · diodel
Params1.88B
Context256K
Best fitQ4_K_M 2.2 GB
DQ
dqnMath v1
Dqnmath · mradermacher
Params4.02B
Context256K
Best fitIQ4_XS 4.0 GB
PY
Pyrz 8B Instruct Uncensored i1
Pyrz · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · second-state
Params1.54B
Context32K
Best fitF16 3.8 GB
OZ
Ozyvler Ozygram Neural 7B Coder MCTS
Ozyvler · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
QW
Qwen3 Esper3 Reasoning CODER Instruct 12B Brainstorm20x
Qwen · mradermacher
Params11.9B
Context40K
Q4_K_M7.2 GB
QW
Qwen3 VL 12B Instruct Brainstorm20x
Qwen · mradermacher
Params11.9B
Context256K
Q4_K_M7.2 GB
KA
Karnak
Karnak · mradermacher
Params40.7B
Context256K
Q4_K_M24.7 GB
OP
OpenELM 450M Instruct
Openelm · mradermacher
Params457M
Context2K
Best fitF16 1.5 GB
QW
Qwen3.8-27B-DFlash2
Qwen · HermiHg
Params1.92B
Context256K
Best fitQ2_K_S 1.2 GB
LO
Loopl Vl 4b
Loopl · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
HO
Holo2 235B A22B
Holo · mradermacher
Params235B
Context256K
Q4_K_M142.2 GB
EU
EuroLLM 9B MeditronFO
Eurollm · mradermacher
Params9.15B
Context4K
Q4_K_M5.6 GB
SP
Spark-X2.5-1.7B
Spark · Mungert
Params1.71B
Context1024K
Best fitBF16 3.7 GB
QW
Qwen3 30B A3B Instruct 2507 REAM Heretic
Qwen · mradermacher
Params23.3B
Context256K
Q4_K_M14.2 GB
QW
Qwen3 235B A22B Thinking 2507
Qwen · bartowski
Params235B
Context256K
Q4_K_M142.6 GB
CY
Cydonia 24B v4.3 Heretic v3
Cydonia · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB