GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2-1.5B
Qwen · legraphista
Params1.54B
Context128K
Best fitBF16 3.8 GB
GGUF
BO
prism ml Bonsai 8B unpacked
Bonsai · bartowski
Params8.19B
Context64K
Q4_K_M5.2 GB
GGUF
LL
Llama 3.2 1b Abliterated
Llama · mradermacher
Params1.24B
Context128K
Best fitQ6_K 1.8 GB
GGUF
LL
Reflection Llama 3.1 8B i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
LY
LYNX ALPHA
Lynx · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
LF
Lfm2.5 2.6b Fable5 Coding Agent Heretic
Lfm · mradermacher
Params2.7B
Context128K
Best fitQ3_K_L 4.0 GB
Tool use
VN
Vnpen Writer 9b v1.0
Vnpen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
DE
DeepSeek R1 Distill Qwen 7B Abliterated v2
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
Reasoning
LL
Llama Guard 3 8B
Llama · legraphista
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
DA
DataPilot ArrowPro 7B KUJIRA
Datapilot · mmnga
Params7.24B
Context4K
Best fitIQ2_S 3.9 GB
GGUF
ST
L3-8B-Stheno-v3.3-32K
Stheno · DavidAU
Params8B
Context32K
Best fitIQ1_M 3.7 GB
Roleplay
CO
Cosmos Reason2 2b
Cosmos · apolo13x
Params2.03B
Context256K
Best fitQ8_0 3.6 GB
ReasoningVision
ST
Starling-LM-7B-alpha-x2
Starling · mradermacher
Params7.24B
Context8K
Best fitIQ2_S 3.9 GB
GGUF
GR
Granite 4.0 H Tiny
Granite · bartowski
Params6.94B
Context1024K
Q4_K_M4.3 GB
GGUF
PY
Pythia 1b
Pythia · mradermacher
Params1.01B
Context2K
Best fitQ6_K 2.4 GB
GGUF
PR
Privacy Filter
Privacy · LocalAI-io
Params1.4B
Context128K
Best fitF16 3.4 GB
GGUF
GE
Gemma-4-E4B-It
Gemma · ente-ai
Params7.5B
Context128K
Best fitBF16 2.1 GB
Vision
AS
AstaBrief_8B
Astabrief · mradermacher
Params8.2B
Context40K
Best fitIQ1_M 4.0 GB
GGUF
MI
MiniCPM5 2B DSpark
MiniCPM · openbmb
Params324M
Context128K
Smallest0.7 GB
CodingTool use
QW
Qwen3.6 35B A3B Uncensored Heretic Native MTP Preserved
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
GGUF
SP
Spark X2.6 4B Uncensored GSQ RCO
Spark · abenzerps
Params4.11B
Context1024K
Best fitIQ3_M 4.0 GB
ReasoningTool use
QW
Qwen3.8 4B Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
GGUF
PY
Pythia 410m Deduped
Pythia · mradermacher
Params405M
Context2K
Best fitQ4_0 1.6 GB
GGUF
QW
Qwen3.8 2B Distill Heretic
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
ReasoningTool use
MI
Voxtral Small 24B 2507
Mistral · bartowski
Params23.6B
Context128K
Q4_K_M14.3 GB
Vision
AU
Audar ASR V1 Turbo
Audar · audarai
Params2.03B
Context64K
Best fitQ8_0 3.6 GB
CodingVision
QW
Qwen3 8B Socratic SFT
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
GGUF
GE
Skaldic Gemma 4 E4B
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
GGUF
QW
Qwen2-1.5B-Instruct
Qwen · legraphista
Params1.54B
Context32K
Best fitBF16 3.8 GB
GGUF
JA
Jais-3-7B
Jais · mradermacher
Params9B
Context512K
Q4_K_M5.6 GB
Reasoning

Other memory sizes

Keep browsing