GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
LL
Home Llama 3.2 3B
Llama · acon96
Params3.21B
Context128K
Best fitQ5_K_M 3.8 GB
YI
Yi-6B-200K
Yi · duyntnet
Params6.06B
Context195K
Best fitQ3_K_M 4.0 GB
DA
danube2-1.8b-WizardLM-Evol-V2-Unfiltered
Danube · mradermacher
Params1.83B
Context8K
Best fitQ8_0 3.0 GB
AG
L3.1 Aglow Vulca v0.1 8B i1
Aglow · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
PA
MS3.2 PaintedFantasy v3 24B
Paintedfantasy · bartowski
Params23.6B
Context128K
Q4_K_M14.3 GB
SM
SmolLM2 1.7B Instruct
SmolLM · unsloth
Params1.71B
Context8K
Best fitQ8_0 3.9 GB
SA
Salience 1.5 Pro
Salience · bartowski
Params35.5B
Context256K
Q4_K_M21.9 GB
FA
Falcon H1 7B Instruct
Falcon · mradermacher
Params7.59B
Context256K
Best fitIQ3_XXS 3.9 GB
QW
Qwen3.5 9B Coder
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
QW
MathSmith HC Problem Synthesizer Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
NE
NeoHorse 1 9B Heretic
Neohorse · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
AB
Absolute_Zero_Reasoner Coder 3b
Absolute · bartowski
Params3.09B
Context32K
Best fitQ6_K_L 3.4 GB
QW
Qwen Image 2.1 PE T2I Pocket 0.8B
Qwen · ML-Intern-lab
Params752M
Context256K
Best fitQ8_0 1.7 GB
WA
WazoCore 8B v6 Long
Wazocore · mradermacher
Params8.15B
Context128K
Best fitIQ2_XXS 4.0 GB
MO
Mochi 4B
Mochi · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
GR
Granite-4.0-H-350M
Granite · unsloth
Params340M
Context1024K
Best fitBF16 2.0 GB
GE
Gemma 2 2B ArliAI RPMax v1.1
Gemma · bartowski
Params2.61B
Context8K
Best fitQ6_K_L 3.7 GB
QW
PAW 27B X3
Qwen · lackonendes
Params4.85B
Context256K
Best fitQ2_K 1.4 GB
GE
Gemma 3 12b It Vl Minimax M2.1 Heretic Uncensored Thinking
Gemma · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
NE
Next 12b
Next · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
MU
Muse Glimmer 30B Uncensored Heretic
Muse · mradermacher
Params27.9B
Context128K
Q4_K_M16.9 GB
AU
Aurora Code 1
Aurora · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
QU
QuasiStarSynth 12B Noslop Absolute Heresy
Quasistarsynth · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
LL
Mahou-1.3b-llama3-8B
Llama · mradermacher
Params8.03B
Context8K
Best fitIQ2_XXS 4.0 GB
IN
Indian Legal Assitant i1
Indian · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
QW
MathSmith HC Qwen3 8B Qwen2_5 32B ShortCoT
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
KU
KURE v3
Kure · mradermacher
Params1.14B
Context256K
Best fitQ6_K 2.0 GB
ST
Starcoder2 3b
StarCoder · QuantFactory
Params3.03B
Context16K
Best fitQ8_0 4.0 GB
DR
Dream v0 Instruct 7B
Dream · bartowski
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
QW
Qwen3 Coder Next Base
Qwen · mradermacher
Params79.7B
Context256K
Q4_K_M48.5 GB