GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 Coder 0.5B Instruct
Qwen · unsloth
Params494M
Context32K
Best fitF16 1.6 GB
Coding
MI
Ministral-3-14B-Nymphaea-RP
Mistral · mradermacher
Params13.5B
Context256K
Q4_K_M8.2 GB
Roleplay
QQ
qqWen 32B Pretrain
Qqwen · mradermacher
Params32.8B
Context32K
Q4_K_M19.9 GB
GGUF
HO
Holo4 27B
Holo · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF
SM
SmolLM 360M Instruct
SmolLM · mradermacher
Params362M
Context2K
Best fitF16 1.6 GB
Tool use
EU
Eubiota Planner 8b
Eubiota · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
ReasoningTool use
QW
Qwen2.5 Coder 0.5B Instruct
Qwen · lmstudio-community
Params494M
Context32K
Best fitQ6_K 1.1 GB
Coding
HY
Hypnos I1 8B
Hypnos · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
ReasoningRoleplay
QW
Qwopus3.5 4B Coder
Qwen · Jackrong
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
CodingReasoningTool use
QW
MiMo-V2.6-Distill-Qwen-9B
Qwen · ngquocvinh
Params8.95B
Context256K
Best fitQ1_0 3.5 GB
CodingTool useVision
GR
Granite 4.1 8b Abliterated
Granite · mradermacher
Params8.38B
Context128K
Best fitIQ1_M 4.0 GB
GGUF
MA
Magidonia 24B v4.3 Absolute Heresy
Magidonia · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
GE
Gemma 4 31B It DFlash
Gemma · Anbeeld
Params1.54B
Context256K
Best fitBF16 3.8 GB
GGUF
MI
MiMo VL 7B RL 2508
Mimo · bartowski
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
NE
Nemotron Mini 4B Instruct
Nemotron · bartowski
Params4.19B
Context4K
Best fitQ3_K_L 4.0 GB
GGUF
QW
Qwen2 VL 7B Instruct Abliterated
Qwen · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
Vision
QW
Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16
Qwen · nuofang
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
LI
LittleInstructionMaker 4B v0.1
Littleinstructionmaker · mradermacher
Params3.96B
Context8K
Best fitQ5_K_S 4.0 GB
Tool use
PO
Polaris VGA 9B Post1.0e
Polaris · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
QW
Qwen3 4B Instruct 2507
Qwen · a-ivanovitch
Params4B
Context256K
Best fitQ3_K_M 3.8 GB
Tool use
PE
Peach 2.0 9B 8k Roleplay
Peach · mradermacher
Params8.83B
Context16K
Best fitIQ2_XS 4.0 GB
Roleplay
QW
Qwen2.5 3B Instruct
Qwen · QuantFactory
Params3.1B
Context32K
Best fitQ6_K 3.3 GB
GGUF
LL
Llama 3.1 8B MultiReflection Instruct i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
LL
Llama3.2 3B Explained
Llama · theprint
Params3.2B
Context128K
Best fitQ5_K_M 3.8 GB
GGUF
AR
ARK 270M Instruct i1
Ark · usermma
Params252M
Context2K
Best fitQ5_K_S 1.0 GB
GGUF
MI
dolphin-2.9.3-mistral-7B-32k
Mistral · bartowski
Params7.25B
Context32K
Best fitIQ2_S 3.9 GB
CodingTool use
MI
cerebras/MiniMax-M2-REAP-139B-A10B
Minimax · bartowski
Params139B
Context192K
Q4_K_M84.4 GB
GGUF
GE
Amoral Gemma3 4B
Gemma · bartowski
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
GGUF
KR
Kronumos
Kronumos · mradermacher
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
Coding
BA
bagel 8b v1.0 i1
Bagel · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
CodingRoleplayTool use

Other memory sizes

Keep browsing