GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
LF
LFM3 1.7B
Lfm · bartowski
Params1.73B
Context256K
Best fitQ8_0 4.0 GB
CodingReasoningTool use
QW
Qwen3.5 9B Heretic v2
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
HE
Hermes 4 14B OBLITERATED V2
Hermes · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
ReasoningTool use
SP
Spark X2.6 4B Heretic
Spark · mradermacher
Params4.11B
Context1024K
Best fitQ3_K_M 3.9 GB
ReasoningTool use
DE
Qwen2.5 MOE 2X7B DeepSeek Abliterated UNCensored 19B i1
DeepSeek · mradermacher
Params19B
Context128K
Q4_K_M11.6 GB
ReasoningTool use
KA
KangalKhan RawRuby 7B i1
Kangalkhan · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
FI
Fino1 8B
Fino · mradermacher
Params8.03B
Context128K
Best fitIQ2_XXS 4.0 GB
ReasoningTool use
YI
Yi-6B-200K
Yi · mradermacher
Params6.06B
Context195K
Best fitQ3_K_M 4.0 GB
GGUF
CO
codegemma-1.1-2b
Codegemma · mmnga
Params2.51B
Context8K
Best fitQ8_0 3.3 GB
Coding
IN
Internlm2_5 7b Chat 1m
InternLM · duyntnet
Params7.74B
Context256K
Best fitIQ2_XS 4.0 GB
GGUF
IN
Index Translate 9B
Index · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
GGUF
NE
NeuTTS Nano
Neutts · BricksDisplay
Params262M
Context2K
Best fitF32 1.7 GB
GGUF
CA
Carnice MoE 35B A3B
Carnice · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
Tool use
WO
WorldSim Opus 3.6 35B A3B
Worldsim · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningRoleplay
QW
Qwen3 4B Instruct 2507 Heretic v4
Qwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
GGUF
NI
Q3 Nilcall 32B v0.1
Nilcall · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
ReasoningRoleplayTool use
MI
MiniCPM5 2B heretic
MiniCPM · saidutta69
Params2.52B
Context128K
Best fitQ8_0 3.5 GB
CodingReasoningTool use
TR
Triangulum 5B It
Triangulum · mradermacher
Params5.41B
Context8K
Best fitQ3_K_M 3.9 GB
Reasoning
DE
DeepSeek R1 Distill Qwen 7B Abliterated Obliteratus
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
Reasoning
ST
StartLux-Decision-v1.1-2B-Instant
Startlux · startlux-models
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
Vision
QW
ReSearch Qwen 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
JI
JiRackUltra_14b
Jirackultra · mradermacher
Params14.8B
Context128K
Q4_K_M9.0 GB
ReasoningTool use
GE
lei HuggingFace Gemma 2 9B 4bit Chat Level Measurement Guide 07242024
Gemma · RichardErkhov
Params3.4B
Context—
Best fitQ4_0 3.2 GB
GGUF
GR
Grug 35b v2
Grug · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
ReasoningTool useVision
ER
Ernie-4.5-0.3B-Pt
Ernie · unsloth
Params361M
Context128K
Best fitF16 1.4 GB
GGUF
QW
Qwen3 Yoyo V4 42B A3B Thinking TOTAL RECALL
Qwen · mradermacher
Params42.4B
Context256K
Q4_K_M25.7 GB
CodingReasoning
LL
NousResearch DeepHermes 3 Llama 3 3B Preview
Llama · bartowski
Params3.21B
Context128K
Best fitQ5_K_L 3.9 GB
ReasoningRoleplayTool use
SP
Spark X2.6 4B Tema_Q Abliterated
Spark · mradermacher
Params4.11B
Context1024K
Best fitQ3_K_M 3.9 GB
ReasoningTool use
GE
Gemma 4 E4B It Erotica
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
GGUF
QW
Qwen3 0.6B
Qwen · QuantFactory
Params752M
Context40K
Best fitQ8_0 2.2 GB
GGUF

Other memory sizes

Keep browsing