GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
WA
Wayra-Perplexity-Estimator-55M
Wayra · latam-gpt
Params55M
Context2K
Best fitF32 0.8 GB
LL
Configurable Llama 3.1 8B Instruct i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
ZW
ZwZ2 4B
Zwz · bartowski
Params4.21B
Context256K
Best fitIQ4_XS 4.0 GB
ME
Mellum2 12B A2.5B Thinking
Mellum · bartowski
Params12.1B
Context128K
Q4_K_M8.2 GB
HY
Hy3
bartowski
Params300B
Context256K
Q4_K_M182.2 GB
DE
DeepSeek R1 Distill Qwen 1.5B
DeepSeek · second-state
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
QW
Qwen3 Coder 30B A3B Instruct
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
LL
Bio Medical Llama 3 8B i1
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
SE
Serenity 27B
Serenity · ReadyArt
Params27.3B
Context256K
Q4_K_M16.8 GB
QW
Qwen3Vl 2b Thinking
Qwen · Qwen
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
MI
MiniCPM V 4.6
MiniCPM · ggml-org
Params752M
Context256K
Best fitBF16 2.4 GB
QW
Qwen2.5 Math 1.5B Instruct
Qwen · mradermacher
Params1.54B
Context4K
Best fitQ6_K 2.0 GB
TR
Transliteration 2.0 4B Safetensors
Transliteration · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
BO
Borealis 12b
Borealis · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
LL
Llama 3.2 1B Instruct
Llama · DavidAU
Params1.24B
Context128K
Best fitQ6_K 1.8 GB
CO
ComfyUI
Comfyui · f5aiteam
Params7.62B
Context125K
Best fitQ4_K_S 3.7 GB
MI
Mistral Small 3.2 24B Instruct 2506 Absolute Heresy
Mistral · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
NE
Dory1.0 4B BF16 Llamacpp
Nemotron · cloudnathan5
Params3.98B
Context256K
Q4_K_M2.4 GB
QW
Qwen3.8 27B Ultra Uncensored Heretic Native MTP Preserved
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
LF
LFM3-Audio-2B
Lfm · AtomicChat
Params1.73B
Context125K
Best fitQ8_0 4.0 GB
CY
Cydonia R1 24B v4
Cydonia · bartowski
Params23.6B
Context128K
Q4_K_M14.3 GB
QW
Qwen3.5 0.8B GPT 6 Astra Luna Fable 5.1 Opus 5.5 distill
Qwen · 0xSojalSec
Params752M
Context256K
Best fitQ4_K_M 1.4 GB
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
FR
FrogNano 4B 2609
Frognano · mradermacher
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
DE
DeepSeek R1 0528 Qwen3 8B Heretic
DeepSeek · mradermacher
Params8.19B
Context128K
Best fitIQ1_M 4.0 GB
GE
gemma 4 12B it qat MTP
Gemma · magnitudedev
Params423M
Context256K
Best fitQ4_0 1.8 GB
QW
OpenSpatial Qwen2.5 VL 7B
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
LL
llama nemotron embed 1b v2
Llama · mradermacher
Params1.24B
Context128K
Best fitF16 3.2 GB
LF
LFM2 12B A1B SpeedDemon The Deckard II HERETIC Uncensored
Lfm · mradermacher
Params11.3B
Context125K
Q4_K_M6.8 GB
SH
Shisutemu Masuta Q3 32B
Shisutemu · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB