GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
QW
Qwen3 4B Thinking Heretic
Qwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
QW
Qwen3 VisionCaption 2B
Qwen · prithivMLmods
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
RA
RakutenAI 2.0 Mini Instruct
Rakutenai · mmnga
Params1.53B
Context128K
Best fitQ8_0 2.5 GB
YI
Yi 6B i1
Yi · mradermacher
Params6.06B
Context4K
Best fitQ3_K_M 4.0 GB
ST
starcoder2-3b
StarCoder · second-state
Params3.03B
Context16K
Best fitQ8_0 4.0 GB
OP
OpenELM 1_1B Instruct
Openelm · mradermacher
Params1.08B
Context2K
Best fitF16 2.9 GB
GR
granite-4.0-micro-base
Granite · Mungert
Params3.4B
Context128K
Q4_K_M2.1 GB
OU
Oumuamua-7b-instruct-v2
Oumuamua · mradermacher
Params7.33B
Context32K
Best fitIQ2_S 3.9 GB
QW
Qwen2.5 Coder 3B Instruct heretic
Qwen · saidutta69
Params3.1B
Context32K
Best fitQ6_K 3.3 GB
NO
Nomic Embed Text V2
Nomic · ggml-org
Params475M
Context2K
Best fitQ8_0 1.3 GB
QW
Einstein-v7-Qwen2-7B
Qwen · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
LF
Lfm2.5-Vl-1.6B
Lfm · unsloth
Params1.17B
Context125K
Best fitBF16 3.9 GB
YI
Yi 9B i1
Yi · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
QW
Qwen3.8 27B Uncensored
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
OS
Osmosis Mcp 4b
Osmosis · osmosis-ai
Params4.41B
Context40K
Best fitQ3_K_M 4.0 GB
SN
Snowflake Arctic Embed L v2.0
Snowflake · Casual-Autopsy
Params567M
Context8K
Best fitF32 3.6 GB
NE
Neuron 46x4B Instruct
Neuron · mradermacher
Params125B
Context256K
Q4_K_M75.8 GB
FL
Flint 40B A2.5B Abliterated
Flint · mradermacher
Params39.8B
Context256K
Q4_K_M24.2 GB
FA
Fara 7B
Fara · bartowski
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
DE
DeepSeek V4 Pro Qwen3.5 4B
DeepSeek · mradermacher
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
QW
Qwen2.5 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
MI
YuisekinAI Mistral 0.7B
Mistral · RichardErkhov
Params746M
Context4K
Best fitQ8_0 1.5 GB
CO
Yi Coder 1.5B
Coder · QuantFactory
Params1.48B
Context128K
Best fitQ8_0 3.7 GB
QW
Qwen3 0.6B Distilled 30B A3B Thinking SFT
Qwen · reaperdoesntknow
Params752M
Context40K
Best fitF16 2.9 GB
QW
Qwen2.5 VL 7B Instruct
Qwen · second-state
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
RU
RusMind 4.5.4 Momentum
Rusmind · DICPlay
Params4.65B
Context128K
Q4_K_M3.4 GB
TA
UI TARS 2B SFT
Tars · bartowski
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
DO
Dots.Mocr
Dots · prithivMLmods
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
GR
Granite 3b Code Base 128k
Granite · mradermacher
Params3.48B
Context125K
Best fitIQ1_S 4.0 GB
HE
Hemlock Philter 14B
Hemlock · mradermacher
Params14.8B
Context32K
Q4_K_M9.0 GB