GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
LF
LFM2.5 24B A2B
Lfm · bartowski
Params24B
Context128K
Q4_K_M15.4 GB
ReasoningTool use
LA
Laguna XS 2.1
Laguna · bartowski
Params33.4B
Context256K
Q4_K_M20.5 GB
GGUF
QW
Qwen3-Vl-8B-Thinking
Qwen · unsloth
Params8.19B
Context256K
Best fitIQ1_S 4.0 GB
ReasoningVision
QW
Qwen3 VL 2B Instruct
Qwen · bartowski
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
Vision
GE
Gemma 4 E2B It Heretic v1
Gemma · mradermacher
Params4.63B
Context128K
Best fitQ2_K_S 4.0 GB
Vision
QW
Qwen3.5 397B A17B
Qwen · bartowski
Params403B
Context256K
Q4_K_M248.3 GB
Vision
VE
Veloxy Titan 32B Vision
Veloxy · mradermacher
Params26.9B
Context256K
Q4_K_M16.5 GB
CodingReasoningVision
ON
OneJev 9B
Onejev · bartowski
Params8.95B
Context256K
Q4_K_M5.8 GB
Vision
QW
Qwen3 Reranker 0.6B GGUF llama cpp
Qwen · Voodisss
Params596M
Context40K
Best fitF16 2.6 GB
Embedding
OR
Ornith 1.0 35B
Ornith · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
Vision
PR
Privacy Filter Multilingual
Privacy · LocalAI-io
Params1.4B
Context128K
Best fitF16 3.4 GB
GGUF
BG
bge m3
BGE · lm-kit
Params567M
Context8K
Best fitF16 2.5 GB
Embedding
QW
Qwen2.5 Coder 7B Instruct
Qwen · unsloth
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
Coding
QW
Qwen.Qwen3 VL Embedding 2B
Qwen · DevQuasar
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
EmbeddingVision
CY
CyberSecQwen 4B
Cybersecqwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
GGUF
PH
gretelai Phi 3 mini 128k instruct
Phi · RichardErkhov
Params1.9B
Context—
Best fitQ2_K 1.3 GB
GGUF
DE
DeepSeek V3.1
DeepSeek · bartowski
Params670B
Context160K
Q4_K_M409.2 GB
GGUF
LF
lfm2.5 2.6b fable5 coding agent heretic
Lfm · saidutta69
Params2.7B
Context128K
Best fitQ3_K_L 4.0 GB
ReasoningTool use
HU
Huihui Ornith 1.5 35B A3B Abliterated
Huihui · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
GGUF
GE
Huihui gemma 4 E2B it qat q4 0 unquantized abliterated
Gemma · huihui-ai
Params4.63B
Context128K
Best fitQ4_0 4.0 GB
Vision
LL
Llama 3.2 1B Instruct
Llama · lmstudio-community
Params1.24B
Context128K
Best fitQ8_0 2.1 GB
GGUF
GE
Gemma 3 4b It Abliterated
Gemma · bartowski
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
Vision
LI
LightOnOCR 2 1B
Lightonocr · ggml-org
Params596M
Context16K
Best fitF16 2.6 GB
Vision
FU
Functiongemma-270M-It
Functiongemma · unsloth
Params268M
Context32K
Best fitBF16 1.2 GB
GGUF
ON
OneJev 4B
Onejev · bartowski
Params4.84B
Context256K
Best fitQ3_K_M 4.0 GB
Vision
LI
LightOnOCR 3 4B
Lightonocr · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
Vision
NE
Nex N2.5 Mini
Nex · bartowski
Params34.7B
Context256K
Q4_K_M22.3 GB
Vision
HO
K2-Horizon-32B
Horizon · mradermacher
Params34.8B
Context512K
Q4_K_M21.1 GB
GGUF
QW
huihui ai Huihui Qwen3.5 0.8B abliterated Ultimate Offensive Red Team
Qwen · azminawwar
Params773M
Context256K
Best fitBF16 2.5 GB
Vision
AP
Apertus-8B-Instruct-2509
Apertus · unsloth
Params8.05B
Context64K
Best fitIQ1_M 3.9 GB
GGUF

Other memory sizes

Keep browsing