GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
QW
Qwen2.5 Coder 0.5B Instruct
Qwen · unsloth
Params494M
Context32K
Best fitF16 1.6 GB
MI
Ministral-3-14B-Nymphaea-RP
Mistral · mradermacher
Params13.5B
Context256K
Q4_K_M8.2 GB
QQ
qqWen 32B Pretrain
Qqwen · mradermacher
Params32.8B
Context32K
Q4_K_M19.9 GB
HO
Holo4 27B
Holo · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
SM
SmolLM 360M Instruct
SmolLM · mradermacher
Params362M
Context2K
Best fitF16 1.6 GB
EU
Eubiota Planner 8b
Eubiota · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
QW
Qwen2.5 Coder 0.5B Instruct
Qwen · lmstudio-community
Params494M
Context32K
Best fitQ6_K 1.1 GB
HY
Hypnos I1 8B
Hypnos · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwopus3.5 4B Coder
Qwen · Jackrong
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
QW
MiMo-V2.6-Distill-Qwen-9B
Qwen · ngquocvinh
Params8.95B
Context256K
Best fitQ1_0 3.5 GB
GR
Granite 4.1 8b Abliterated
Granite · mradermacher
Params8.38B
Context128K
Best fitIQ1_M 4.0 GB
MA
Magidonia 24B v4.3 Absolute Heresy
Magidonia · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GE
Gemma 4 31B It DFlash
Gemma · Anbeeld
Params1.54B
Context256K
Best fitBF16 3.8 GB
MI
MiMo VL 7B RL 2508
Mimo · bartowski
Params7.62B
Context125K
Q4_K_M4.7 GB
NE
Nemotron Mini 4B Instruct
Nemotron · bartowski
Params4.19B
Context4K
Best fitQ3_K_L 4.0 GB
QW
Qwen2 VL 7B Instruct Abliterated
Qwen · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
QW
Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16
Qwen · nuofang
Params8.95B
Context256K
Q4_K_M5.6 GB
LI
LittleInstructionMaker 4B v0.1
Littleinstructionmaker · mradermacher
Params3.96B
Context8K
Best fitQ5_K_S 4.0 GB
PO
Polaris VGA 9B Post1.0e
Polaris · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen3 4B Instruct 2507
Qwen · a-ivanovitch
Params4B
Context256K
Best fitQ3_K_M 3.8 GB
PE
Peach 2.0 9B 8k Roleplay
Peach · mradermacher
Params8.83B
Context16K
Best fitIQ2_XS 4.0 GB
QW
Qwen2.5 3B Instruct
Qwen · QuantFactory
Params3.1B
Context32K
Best fitQ6_K 3.3 GB
LL
Llama 3.1 8B MultiReflection Instruct i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
LL
Llama3.2 3B Explained
Llama · theprint
Params3.2B
Context128K
Best fitQ5_K_M 3.8 GB
AR
ARK 270M Instruct i1
Ark · usermma
Params252M
Context2K
Best fitQ5_K_S 1.0 GB
MI
dolphin-2.9.3-mistral-7B-32k
Mistral · bartowski
Params7.25B
Context32K
Best fitIQ2_S 3.9 GB
MI
cerebras/MiniMax-M2-REAP-139B-A10B
Minimax · bartowski
Params139B
Context192K
Q4_K_M84.4 GB
GE
Amoral Gemma3 4B
Gemma · bartowski
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
KR
Kronumos
Kronumos · mradermacher
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
BA
bagel 8b v1.0 i1
Bagel · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB