GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
AN
AnLLM-EP-xllarge-wikiart
Anllm · mradermacher
Params368M
Context4K
Best fitF16 1.8 GB
MI
MiniCPM V 4.7 1B
MiniCPM · prithivMLmods
Params1.01B
Context256K
Best fitBF16 2.9 GB
LL
Llama32 1b Llm Abliteration
Llama · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
QW
TinySwallow 1.5B Instruct
Qwen · SakanaAI
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
QW
Josiefied Qwen2.5 7B Instruct Abliterated v2
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
NE
Nex N2.5 Mini
Nex · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
LL
Llama-3-8B-Irene-v0.1
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
QW
Bella Tao Merged Qwen2_5 Coder 7b
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
QW
UmarTransit 1B
Qwen · umarfarookm
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
OP
openchat 3.5 i1
Openchat · mradermacher
Params7.24B
Context8K
Best fitIQ2_S 3.9 GB
QW
QwenDGAF Coder 3B
Qwen · hecan3d
Params3.1B
Context32K
Best fitQ6_K 3.3 GB
QW
Qwen3.6 28B REAP
Qwen · mradermacher
Params28.2B
Context256K
Q4_K_M17.3 GB
KU
KukulStanta InfinityRP 7B slerp i1
Kukulstanta · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GE
Gemma 3 Uncensored_v2 Merged
Gemma · mradermacher
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
GR
Grug 9b
Grug · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
EM
Emma 6 Hf
Emma · PaoloneSborracchione
Params1.55B
Context4K
Best fitQ8_0 2.7 GB
HU
Huihui Hunyuan MT Chimera 7B Abliterated
Huihui · mradermacher
Params7.5B
Context256K
Best fitIQ2_XS 4.0 GB
CL
Claire 7B 0.1
Claire · mradermacher
Params7.22B
Context2K
Best fitIQ2_XXS 4.0 GB
WI
WizardMath 7B V1.1 i1
Wizardmath · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
QW
Qwen2.5 7B Instruct Uncensored
Qwen · Andycurrent
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
DA
danube2-1.8b-Synthia-v1.3
Danube · mradermacher
Params1.83B
Context8K
Best fitQ6_K 2.5 GB
QW
Qwen3.5 40B Claude 4.6 Opus Deckard Heretic Uncensored Thinking
Qwen · mradermacher
Params39.1B
Context256K
Q4_K_M23.9 GB
IC
Iced-Lemon-Cookie-7B
Iced · backyardai
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
QW
Qwen3 VL 2B Instruct Abliterated v1
Qwen · prithivMLmods
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
TE
test gguf models
Test · ngxson
Params2B
Context32K
Best fitQ4_K_M 3.3 GB
MI
Ministral 3 8B Base 2512
Mistral · mradermacher
Params8.49B
Context256K
Best fitIQ1_M 3.9 GB
IC
IceTea21EnergyDrinkRPV13 dpo240 i1
Icetea · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
QW
qwen 2.5 coder Uncensored
Qwen · TobiasLogic
Params1.54B
Context32K
Best fitQ4_K_M 1.7 GB
LL
Llama OuteTTS 1.0 1B
Llama · OuteAI
Params1.25B
Context128K
Q4_K_M0.8 GB
MI
Mistral v0.2 Nexus Internal Knowledge Map 7B i1
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB