GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5 9B Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ1_S 4.0 GB
GGUF
CH
Chaotic Order 24B V3
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
GP
gpt2
Gpt · QuantFactory
Params163M
Context1K
Best fitQ8_0 1.0 GB
GGUF
QW
Qwen3 2507 4B Instruct Haiku 4.5 Merged
Qwen · bartowski
Params4B
Context256K
Best fitQ3_K_L 4.0 GB
GGUF
IR
Irene-RP-v2-7B
Irene · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
Roleplay
QW
Qwen2 1.5B Instruct
Qwen · Goekdeniz-Guelmez
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
GGUF
MI
MiniMax M3
Minimax · bartowski
Params426B
Context1024K
Q4_K_M261.3 GB
Vision
SE
Seed OSS 36B Instruct
Seed · bartowski
Params36.2B
Context512K
Q4_K_M21.8 GB
GGUF
QW
Qwen3 0.6B Abliterated
Qwen · bartowski
Params596M
Context40K
Best fitBF16 2.6 GB
GGUF
SO
Solar Open 100B
Solar · mradermacher
Params103B
Context128K
Q4_K_M62.3 GB
GGUF
RH
rho-math-1b-interpreter-v0.1
Rho · RichardErkhov
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
GGUF
QW
Qwen2.5 1.5B Instruct
Qwen · QuantFactory
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
GGUF
QW
Qwen3.6 35B A3B DFlash2
Qwen · aminya
Params526M
Context256K
Best fitBF16 1.8 GB
GGUF
KA
Kadoe 4B
Kadoe · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
ReasoningTool useVision
WI
Winterreise-m7
Winterreise · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
OC
Occamy 1.0
Occamy · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
Tool useVision
GL
Poe 8B GLM5 Opus4.6 Sonnet4.5 Kimi Grok Gemini 3 Pro Preview HERETIC
GLM · mradermacher
Params8.19B
Context256K
Best fitIQ1_M 4.0 GB
Vision
LL
Llama 3.2 3B MoE 4Expert
Llama · mradermacher
Params9.55B
Context128K
Best fitIQ1_M 3.8 GB
Coding
MI
Ministral 3b instruct
Mistral · mradermacher
Params3.32B
Context128K
Best fitQ6_K 3.7 GB
GGUF
QW
Swift 1.5 Qwen3.8 27B Uncensored MTP
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
GGUF
PY
Pythia 1.4b
Pythia · mradermacher
Params1.41B
Context2K
Best fitQ6_K 3.3 GB
GGUF
SP
SpatialAxiom 9B
Spatialaxiom · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Vision
DE
DeepSeek R1 Distill Qwen 1.5B Heretic
DeepSeek · saidutta69
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
ReasoningTool use
DE
deepseek coder 1.3B kexer
DeepSeek · QuantFactory
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
Coding
QW
MiRL Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
ReasoningTool use
TI
TinyLlama 1.1B Chat v1.0 imatrix
Tinyllama · duyntnet
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
GGUF
QW
Qwen3.5 2B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
ReasoningRoleplay
GE
Gemma 3 1b It Qat Q4_0 Unquantized
Gemma · ggml-org
Params1B
Context32K
Best fitQ4_0 1.4 GB
GGUF
OV
OvisOCR2
Ovisocr · MDfox
Params752M
Context256K
Best fitBF16 2.4 GB
Vision
LI
LightOnOCR 3 0.8B
Lightonocr · mradermacher
Params752M
Context256K
Best fitF16 2.4 GB
Vision

Other memory sizes

Keep browsing