GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
BE
Behemoth X 123B v2.1
Behemoth · bartowski
Params123B
Context128K
Q4_K_M73.2 GB
GGUF
RO
Rocinante X 12B v1 Heretic Uncensored
Rocinante · mradermacher
Params12.2B
Context128K
Q4_K_M7.5 GB
Roleplay
GR
granite docling 258M
Granite · ggml-org
Params164M
Context8K
Best fitF16 1.0 GB
Vision
AV
Average_Normie_v3.69_8B
Average · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
IN
InternVL3 2B Instruct
Internvl · ggml-org
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
Vision
NU
NuMarkdown-8B-Thinking
Numarkdown · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
ReasoningVision
QW
Qwen3.5 9B DFlash
Qwen · Anbeeld
Params1.29B
Context256K
Best fitBF16 3.3 GB
CodingEmbedding
QW
Qwen2.5 7B Instruct
Qwen · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
WR
Wraith 8b
Wraith · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
ReasoningRoleplay
QW
Qwen3.5 2B GPT 5.1 HighIQ INSTRUCT
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
Reasoning
MU
Muslim 6B v3
Muslim · mradermacher
Params5.94B
Context256K
Best fitIQ1_M 3.9 GB
Tool use
GE
Gemma 4 E4B It GMCE
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
GGUF
NE
Next2.5
Next · thelamapi
Params4.2B
Context256K
Best fitIQ4_XS 4.0 GB
ReasoningVision
QW
Qwen3 5 2b
Qwen · diodel
Params1.88B
Context256K
Best fitQ4_K_M 2.2 GB
GGUF
DQ
dqnMath v1
Dqnmath · mradermacher
Params4.02B
Context256K
Best fitIQ4_XS 4.0 GB
ReasoningTool use
PY
Pyrz 8B Instruct Uncensored i1
Pyrz · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · second-state
Params1.54B
Context32K
Best fitF16 3.8 GB
Coding
OZ
Ozyvler Ozygram Neural 7B Coder MCTS
Ozyvler · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
Coding
QW
Qwen3 Esper3 Reasoning CODER Instruct 12B Brainstorm20x
Qwen · mradermacher
Params11.9B
Context40K
Q4_K_M7.2 GB
CodingReasoning
QW
Qwen3 VL 12B Instruct Brainstorm20x
Qwen · mradermacher
Params11.9B
Context256K
Q4_K_M7.2 GB
CodingVision
KA
Karnak
Karnak · mradermacher
Params40.7B
Context256K
Q4_K_M24.7 GB
Tool use
OP
OpenELM 450M Instruct
Openelm · mradermacher
Params457M
Context2K
Best fitF16 1.5 GB
GGUF
QW
Qwen3.8-27B-DFlash2
Qwen · HermiHg
Params1.92B
Context256K
Best fitQ2_K_S 1.2 GB
GGUF
LO
Loopl Vl 4b
Loopl · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
Tool useVision
HO
Holo2 235B A22B
Holo · mradermacher
Params235B
Context256K
Q4_K_M142.2 GB
GGUF
EU
EuroLLM 9B MeditronFO
Eurollm · mradermacher
Params9.15B
Context4K
Q4_K_M5.6 GB
GGUF
SP
Spark-X2.5-1.7B
Spark · Mungert
Params1.71B
Context1024K
Best fitBF16 3.7 GB
GGUF
QW
Qwen3 30B A3B Instruct 2507 REAM Heretic
Qwen · mradermacher
Params23.3B
Context256K
Q4_K_M14.2 GB
GGUF
QW
Qwen3 235B A22B Thinking 2507
Qwen · bartowski
Params235B
Context256K
Q4_K_M142.6 GB
Reasoning
CY
Cydonia 24B v4.3 Heretic v3
Cydonia · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
Coding

Other memory sizes

Keep browsing