GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
AN
AnLLM-EP-xllarge-wikiart
Anllm · mradermacher
Params368M
Context4K
Best fitF16 1.8 GB
GGUF
MI
MiniCPM V 4.7 1B
MiniCPM · prithivMLmods
Params1.01B
Context256K
Best fitBF16 2.9 GB
Vision
LL
Llama32 1b Llm Abliteration
Llama · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
Tool use
QW
TinySwallow 1.5B Instruct
Qwen · SakanaAI
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
GGUF
QW
Josiefied Qwen2.5 7B Instruct Abliterated v2
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
NE
Nex N2.5 Mini
Nex · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
GGUF
LL
Llama-3-8B-Irene-v0.1
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
QW
Bella Tao Merged Qwen2_5 Coder 7b
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
Coding
QW
UmarTransit 1B
Qwen · umarfarookm
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
GGUF
OP
openchat 3.5 i1
Openchat · mradermacher
Params7.24B
Context8K
Best fitIQ2_S 3.9 GB
Coding
QW
QwenDGAF Coder 3B
Qwen · hecan3d
Params3.1B
Context32K
Best fitQ6_K 3.3 GB
CodingRoleplay
QW
Qwen3.6 28B REAP
Qwen · mradermacher
Params28.2B
Context256K
Q4_K_M17.3 GB
GGUF
KU
KukulStanta InfinityRP 7B slerp i1
Kukulstanta · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
GE
Gemma 3 Uncensored_v2 Merged
Gemma · mradermacher
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
GGUF
GR
Grug 9b
Grug · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
CodingReasoningTool use
EM
Emma 6 Hf
Emma · PaoloneSborracchione
Params1.55B
Context4K
Best fitQ8_0 2.7 GB
Coding
HU
Huihui Hunyuan MT Chimera 7B Abliterated
Huihui · mradermacher
Params7.5B
Context256K
Best fitIQ2_XS 4.0 GB
GGUF
CL
Claire 7B 0.1
Claire · mradermacher
Params7.22B
Context2K
Best fitIQ2_XXS 4.0 GB
GGUF
WI
WizardMath 7B V1.1 i1
Wizardmath · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
QW
Qwen2.5 7B Instruct Uncensored
Qwen · Andycurrent
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
DA
danube2-1.8b-Synthia-v1.3
Danube · mradermacher
Params1.83B
Context8K
Best fitQ6_K 2.5 GB
GGUF
QW
Qwen3.5 40B Claude 4.6 Opus Deckard Heretic Uncensored Thinking
Qwen · mradermacher
Params39.1B
Context256K
Q4_K_M23.9 GB
CodingReasoningRoleplay
IC
Iced-Lemon-Cookie-7B
Iced · backyardai
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
QW
Qwen3 VL 2B Instruct Abliterated v1
Qwen · prithivMLmods
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
Vision
TE
test gguf models
Test · ngxson
Params2B
Context32K
Best fitQ4_K_M 3.3 GB
GGUF
MI
Ministral 3 8B Base 2512
Mistral · mradermacher
Params8.49B
Context256K
Best fitIQ1_M 3.9 GB
GGUF
IC
IceTea21EnergyDrinkRPV13 dpo240 i1
Icetea · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
QW
qwen 2.5 coder Uncensored
Qwen · TobiasLogic
Params1.54B
Context32K
Best fitQ4_K_M 1.7 GB
Coding
LL
Llama OuteTTS 1.0 1B
Llama · OuteAI
Params1.25B
Context128K
Q4_K_M0.8 GB
GGUF
MI
Mistral v0.2 Nexus Internal Knowledge Map 7B i1
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF

Other memory sizes

Keep browsing