GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
LF
LFM3 1.7B
Lfm · bartowski
Params1.73B
Context256K
Best fitQ8_0 4.0 GB
QW
Qwen3.5 9B Heretic v2
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
HE
Hermes 4 14B OBLITERATED V2
Hermes · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
SP
Spark X2.6 4B Heretic
Spark · mradermacher
Params4.11B
Context1024K
Best fitQ3_K_M 3.9 GB
DE
Qwen2.5 MOE 2X7B DeepSeek Abliterated UNCensored 19B i1
DeepSeek · mradermacher
Params19B
Context128K
Q4_K_M11.6 GB
KA
KangalKhan RawRuby 7B i1
Kangalkhan · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
FI
Fino1 8B
Fino · mradermacher
Params8.03B
Context128K
Best fitIQ2_XXS 4.0 GB
YI
Yi-6B-200K
Yi · mradermacher
Params6.06B
Context195K
Best fitQ3_K_M 4.0 GB
CO
codegemma-1.1-2b
Codegemma · mmnga
Params2.51B
Context8K
Best fitQ8_0 3.3 GB
IN
Internlm2_5 7b Chat 1m
InternLM · duyntnet
Params7.74B
Context256K
Best fitIQ2_XS 4.0 GB
IN
Index Translate 9B
Index · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
NE
NeuTTS Nano
Neutts · BricksDisplay
Params262M
Context2K
Best fitF32 1.7 GB
CA
Carnice MoE 35B A3B
Carnice · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
WO
WorldSim Opus 3.6 35B A3B
Worldsim · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen3 4B Instruct 2507 Heretic v4
Qwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
NI
Q3 Nilcall 32B v0.1
Nilcall · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
MI
MiniCPM5 2B heretic
MiniCPM · saidutta69
Params2.52B
Context128K
Best fitQ8_0 3.5 GB
TR
Triangulum 5B It
Triangulum · mradermacher
Params5.41B
Context8K
Best fitQ3_K_M 3.9 GB
DE
DeepSeek R1 Distill Qwen 7B Abliterated Obliteratus
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
ST
StartLux-Decision-v1.1-2B-Instant
Startlux · startlux-models
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
QW
ReSearch Qwen 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
JI
JiRackUltra_14b
Jirackultra · mradermacher
Params14.8B
Context128K
Q4_K_M9.0 GB
GE
lei HuggingFace Gemma 2 9B 4bit Chat Level Measurement Guide 07242024
Gemma · RichardErkhov
Params3.4B
Context—
Best fitQ4_0 3.2 GB
GR
Grug 35b v2
Grug · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
ER
Ernie-4.5-0.3B-Pt
Ernie · unsloth
Params361M
Context128K
Best fitF16 1.4 GB
QW
Qwen3 Yoyo V4 42B A3B Thinking TOTAL RECALL
Qwen · mradermacher
Params42.4B
Context256K
Q4_K_M25.7 GB
LL
NousResearch DeepHermes 3 Llama 3 3B Preview
Llama · bartowski
Params3.21B
Context128K
Best fitQ5_K_L 3.9 GB
SP
Spark X2.6 4B Tema_Q Abliterated
Spark · mradermacher
Params4.11B
Context1024K
Best fitQ3_K_M 3.9 GB
GE
Gemma 4 E4B It Erotica
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
QW
Qwen3 0.6B
Qwen · QuantFactory
Params752M
Context40K
Best fitQ8_0 2.2 GB