GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3 4B Thinking Heretic
Qwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
ReasoningRoleplayTool use
QW
Qwen3 VisionCaption 2B
Qwen · prithivMLmods
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
Vision
RA
RakutenAI 2.0 Mini Instruct
Rakutenai · mmnga
Params1.53B
Context128K
Best fitQ8_0 2.5 GB
GGUF
YI
Yi 6B i1
Yi · mradermacher
Params6.06B
Context4K
Best fitQ3_K_M 4.0 GB
GGUF
ST
starcoder2-3b
StarCoder · second-state
Params3.03B
Context16K
Best fitQ8_0 4.0 GB
Coding
OP
OpenELM 1_1B Instruct
Openelm · mradermacher
Params1.08B
Context2K
Best fitF16 2.9 GB
GGUF
GR
granite-4.0-micro-base
Granite · Mungert
Params3.4B
Context128K
Q4_K_M2.1 GB
GGUF
OU
Oumuamua-7b-instruct-v2
Oumuamua · mradermacher
Params7.33B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
QW
Qwen2.5 Coder 3B Instruct heretic
Qwen · saidutta69
Params3.1B
Context32K
Best fitQ6_K 3.3 GB
CodingRoleplay
NO
Nomic Embed Text V2
Nomic · ggml-org
Params475M
Context2K
Best fitQ8_0 1.3 GB
Embedding
QW
Einstein-v7-Qwen2-7B
Qwen · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
Tool use
LF
Lfm2.5-Vl-1.6B
Lfm · unsloth
Params1.17B
Context125K
Best fitBF16 3.9 GB
Vision
YI
Yi 9B i1
Yi · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
GGUF
QW
Qwen3.8 27B Uncensored
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
Vision
OS
Osmosis Mcp 4b
Osmosis · osmosis-ai
Params4.41B
Context40K
Best fitQ3_K_M 4.0 GB
GGUF
SN
Snowflake Arctic Embed L v2.0
Snowflake · Casual-Autopsy
Params567M
Context8K
Best fitF32 3.6 GB
Embedding
NE
Neuron 46x4B Instruct
Neuron · mradermacher
Params125B
Context256K
Q4_K_M75.8 GB
Tool use
FL
Flint 40B A2.5B Abliterated
Flint · mradermacher
Params39.8B
Context256K
Q4_K_M24.2 GB
ReasoningTool use
FA
Fara 7B
Fara · bartowski
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
Vision
DE
DeepSeek V4 Pro Qwen3.5 4B
DeepSeek · mradermacher
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
ReasoningVision
QW
Qwen2.5 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
GGUF
MI
YuisekinAI Mistral 0.7B
Mistral · RichardErkhov
Params746M
Context4K
Best fitQ8_0 1.5 GB
GGUF
CO
Yi Coder 1.5B
Coder · QuantFactory
Params1.48B
Context128K
Best fitQ8_0 3.7 GB
Coding
QW
Qwen3 0.6B Distilled 30B A3B Thinking SFT
Qwen · reaperdoesntknow
Params752M
Context40K
Best fitF16 2.9 GB
ReasoningTool use
QW
Qwen2.5 VL 7B Instruct
Qwen · second-state
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
Vision
RU
RusMind 4.5.4 Momentum
Rusmind · DICPlay
Params4.65B
Context128K
Q4_K_M3.4 GB
ReasoningTool use
TA
UI TARS 2B SFT
Tars · bartowski
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
Vision
DO
Dots.Mocr
Dots · prithivMLmods
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
CodingVision
GR
Granite 3b Code Base 128k
Granite · mradermacher
Params3.48B
Context125K
Best fitIQ1_S 4.0 GB
Coding
HE
Hemlock Philter 14B
Hemlock · mradermacher
Params14.8B
Context32K
Q4_K_M9.0 GB
GGUF

Other memory sizes

Keep browsing