GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
QW
Qwen3.5 9B Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ1_S 4.0 GB
CH
Chaotic Order 24B V3
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GP
gpt2
Gpt · QuantFactory
Params163M
Context1K
Best fitQ8_0 1.0 GB
QW
Qwen3 2507 4B Instruct Haiku 4.5 Merged
Qwen · bartowski
Params4B
Context256K
Best fitQ3_K_L 4.0 GB
IR
Irene-RP-v2-7B
Irene · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
QW
Qwen2 1.5B Instruct
Qwen · Goekdeniz-Guelmez
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
MI
MiniMax M3
Minimax · bartowski
Params426B
Context1024K
Q4_K_M261.3 GB
SE
Seed OSS 36B Instruct
Seed · bartowski
Params36.2B
Context512K
Q4_K_M21.8 GB
QW
Qwen3 0.6B Abliterated
Qwen · bartowski
Params596M
Context40K
Best fitBF16 2.6 GB
SO
Solar Open 100B
Solar · mradermacher
Params103B
Context128K
Q4_K_M62.3 GB
RH
rho-math-1b-interpreter-v0.1
Rho · RichardErkhov
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
QW
Qwen2.5 1.5B Instruct
Qwen · QuantFactory
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
QW
Qwen3.6 35B A3B DFlash2
Qwen · aminya
Params526M
Context256K
Best fitBF16 1.8 GB
KA
Kadoe 4B
Kadoe · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
WI
Winterreise-m7
Winterreise · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
OC
Occamy 1.0
Occamy · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
GL
Poe 8B GLM5 Opus4.6 Sonnet4.5 Kimi Grok Gemini 3 Pro Preview HERETIC
GLM · mradermacher
Params8.19B
Context256K
Best fitIQ1_M 4.0 GB
LL
Llama 3.2 3B MoE 4Expert
Llama · mradermacher
Params9.55B
Context128K
Best fitIQ1_M 3.8 GB
MI
Ministral 3b instruct
Mistral · mradermacher
Params3.32B
Context128K
Best fitQ6_K 3.7 GB
QW
Swift 1.5 Qwen3.8 27B Uncensored MTP
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
PY
Pythia 1.4b
Pythia · mradermacher
Params1.41B
Context2K
Best fitQ6_K 3.3 GB
SP
SpatialAxiom 9B
Spatialaxiom · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
DE
DeepSeek R1 Distill Qwen 1.5B Heretic
DeepSeek · saidutta69
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
DE
deepseek coder 1.3B kexer
DeepSeek · QuantFactory
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
QW
MiRL Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
TI
TinyLlama 1.1B Chat v1.0 imatrix
Tinyllama · duyntnet
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
QW
Qwen3.5 2B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
GE
Gemma 3 1b It Qat Q4_0 Unquantized
Gemma · ggml-org
Params1B
Context32K
Best fitQ4_0 1.4 GB
OV
OvisOCR2
Ovisocr · MDfox
Params752M
Context256K
Best fitBF16 2.4 GB
LI
LightOnOCR 3 0.8B
Lightonocr · mradermacher
Params752M
Context256K
Best fitF16 2.4 GB