GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
LF
LFM2.5 24B A2B
Lfm · bartowski
Params24B
Context128K
Q4_K_M15.4 GB
LA
Laguna XS 2.1
Laguna · bartowski
Params33.4B
Context256K
Q4_K_M20.5 GB
QW
Qwen3-Vl-8B-Thinking
Qwen · unsloth
Params8.19B
Context256K
Best fitIQ1_S 4.0 GB
QW
Qwen3 VL 2B Instruct
Qwen · bartowski
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
GE
Gemma 4 E2B It Heretic v1
Gemma · mradermacher
Params4.63B
Context128K
Best fitQ2_K_S 4.0 GB
QW
Qwen3.5 397B A17B
Qwen · bartowski
Params403B
Context256K
Q4_K_M248.3 GB
VE
Veloxy Titan 32B Vision
Veloxy · mradermacher
Params26.9B
Context256K
Q4_K_M16.5 GB
ON
OneJev 9B
Onejev · bartowski
Params8.95B
Context256K
Q4_K_M5.8 GB
QW
Qwen3 Reranker 0.6B GGUF llama cpp
Qwen · Voodisss
Params596M
Context40K
Best fitF16 2.6 GB
OR
Ornith 1.0 35B
Ornith · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
PR
Privacy Filter Multilingual
Privacy · LocalAI-io
Params1.4B
Context128K
Best fitF16 3.4 GB
BG
bge m3
BGE · lm-kit
Params567M
Context8K
Best fitF16 2.5 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · unsloth
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
QW
Qwen.Qwen3 VL Embedding 2B
Qwen · DevQuasar
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
CY
CyberSecQwen 4B
Cybersecqwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
PH
gretelai Phi 3 mini 128k instruct
Phi · RichardErkhov
Params1.9B
Context—
Best fitQ2_K 1.3 GB
DE
DeepSeek V3.1
DeepSeek · bartowski
Params670B
Context160K
Q4_K_M409.2 GB
LF
lfm2.5 2.6b fable5 coding agent heretic
Lfm · saidutta69
Params2.7B
Context128K
Best fitQ3_K_L 4.0 GB
HU
Huihui Ornith 1.5 35B A3B Abliterated
Huihui · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
GE
Huihui gemma 4 E2B it qat q4 0 unquantized abliterated
Gemma · huihui-ai
Params4.63B
Context128K
Best fitQ4_0 4.0 GB
LL
Llama 3.2 1B Instruct
Llama · lmstudio-community
Params1.24B
Context128K
Best fitQ8_0 2.1 GB
GE
Gemma 3 4b It Abliterated
Gemma · bartowski
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
LI
LightOnOCR 2 1B
Lightonocr · ggml-org
Params596M
Context16K
Best fitF16 2.6 GB
FU
Functiongemma-270M-It
Functiongemma · unsloth
Params268M
Context32K
Best fitBF16 1.2 GB
ON
OneJev 4B
Onejev · bartowski
Params4.84B
Context256K
Best fitQ3_K_M 4.0 GB
LI
LightOnOCR 3 4B
Lightonocr · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
NE
Nex N2.5 Mini
Nex · bartowski
Params34.7B
Context256K
Q4_K_M22.3 GB
HO
K2-Horizon-32B
Horizon · mradermacher
Params34.8B
Context512K
Q4_K_M21.1 GB
QW
huihui ai Huihui Qwen3.5 0.8B abliterated Ultimate Offensive Red Team
Qwen · azminawwar
Params773M
Context256K
Best fitBF16 2.5 GB
AP
Apertus-8B-Instruct-2509
Apertus · unsloth
Params8.05B
Context64K
Best fitIQ1_M 3.9 GB