GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Home Llama 3.2 3B
Llama · acon96
Params3.21B
Context128K
Best fitQ5_K_M 3.8 GB
GGUF
YI
Yi-6B-200K
Yi · duyntnet
Params6.06B
Context195K
Best fitQ3_K_M 4.0 GB
GGUF
DA
danube2-1.8b-WizardLM-Evol-V2-Unfiltered
Danube · mradermacher
Params1.83B
Context8K
Best fitQ8_0 3.0 GB
GGUF
AG
L3.1 Aglow Vulca v0.1 8B i1
Aglow · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
Roleplay
PA
MS3.2 PaintedFantasy v3 24B
Paintedfantasy · bartowski
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
SM
SmolLM2 1.7B Instruct
SmolLM · unsloth
Params1.71B
Context8K
Best fitQ8_0 3.9 GB
GGUF
SA
Salience 1.5 Pro
Salience · bartowski
Params35.5B
Context256K
Q4_K_M21.9 GB
CodingReasoningTool use
FA
Falcon H1 7B Instruct
Falcon · mradermacher
Params7.59B
Context256K
Best fitIQ3_XXS 3.9 GB
GGUF
QW
Qwen3.5 9B Coder
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningTool use
QW
MathSmith HC Problem Synthesizer Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
GGUF
NE
NeoHorse 1 9B Heretic
Neohorse · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
AB
Absolute_Zero_Reasoner Coder 3b
Absolute · bartowski
Params3.09B
Context32K
Best fitQ6_K_L 3.4 GB
Coding
QW
Qwen Image 2.1 PE T2I Pocket 0.8B
Qwen · ML-Intern-lab
Params752M
Context256K
Best fitQ8_0 1.7 GB
GGUF
WA
WazoCore 8B v6 Long
Wazocore · mradermacher
Params8.15B
Context128K
Best fitIQ2_XXS 4.0 GB
Tool use
MO
Mochi 4B
Mochi · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
GGUF
GR
Granite-4.0-H-350M
Granite · unsloth
Params340M
Context1024K
Best fitBF16 2.0 GB
GGUF
GE
Gemma 2 2B ArliAI RPMax v1.1
Gemma · bartowski
Params2.61B
Context8K
Best fitQ6_K_L 3.7 GB
GGUF
QW
PAW 27B X3
Qwen · lackonendes
Params4.85B
Context256K
Best fitQ2_K 1.4 GB
GGUF
GE
Gemma 3 12b It Vl Minimax M2.1 Heretic Uncensored Thinking
Gemma · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
ReasoningVision
NE
Next 12b
Next · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
Coding
MU
Muse Glimmer 30B Uncensored Heretic
Muse · mradermacher
Params27.9B
Context128K
Q4_K_M16.9 GB
GGUF
AU
Aurora Code 1
Aurora · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
Coding
QU
QuasiStarSynth 12B Noslop Absolute Heresy
Quasistarsynth · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
Mahou-1.3b-llama3-8B
Llama · mradermacher
Params8.03B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
IN
Indian Legal Assitant i1
Indian · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
QW
MathSmith HC Qwen3 8B Qwen2_5 32B ShortCoT
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
GGUF
KU
KURE v3
Kure · mradermacher
Params1.14B
Context256K
Best fitQ6_K 2.0 GB
GGUF
ST
Starcoder2 3b
StarCoder · QuantFactory
Params3.03B
Context16K
Best fitQ8_0 4.0 GB
Coding
DR
Dream v0 Instruct 7B
Dream · bartowski
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
GGUF
QW
Qwen3 Coder Next Base
Qwen · mradermacher
Params79.7B
Context256K
Q4_K_M48.5 GB
Coding

Other memory sizes

Keep browsing