GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 Coder 14B Instruct 3MPER0RR Abliterated
Qwen · mradermacher
Params14.8B
Context32K
Q4_K_M9.0 GB
Coding
BO
Boptruth Agatha 7B i1
Boptruth · mradermacher
Params7.24B
Context128K
Best fitIQ2_S 3.9 GB
GGUF
EM
Emotion Companion 7b
Emotion · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
SM
SmolLM2 135M Instruct
SmolLM · QuantFactory
Params135M
Context8K
Best fitQ2_K 0.8 GB
GGUF
LL
Fireball Meta Llama 3.2 8B Instruct agent 003 128k code DPO i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
Coding
LI
LightOnOCR 3 4B
Lightonocr · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
Vision
QW
Qwen2.5 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
QW
Qwen3.6 35B A3B Abliterated
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
GGUF
QW
Qwen3.5 4B RpRMax v1
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
GGUF
GE
Gemma 3 4b It Abliterated v2
Gemma · mlabonne
Params3.88B
Context128K
Best fitQ3_K_M 3.7 GB
Vision
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · alphaduriendur
Params1.54B
Context32K
Best fitQ4_K_M 1.7 GB
Coding
CH
ChatGPT 5 Q8 0
Chatgpt · Hack337
Params494M
Context32K
Best fitQ8_0 1.1 GB
GGUF
IC
IceEspressoRPv6 i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
GP
gpt2 i1
Gpt · mradermacher
Params163M
Context1K
Best fitIQ1_S 0.9 GB
GGUF
NO
NousCoder 14B
Nouscoder · bartowski
Params14.8B
Context80K
Q4_K_M9.0 GB
Coding
LL
Llama 3.1 8B
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
LL
Llama-3-Instruct-8B-SPPO-Iter3
Llama · legraphista
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
EX
EXAONE 3.5 2.4B Instruct
Exaone · lmstudio-community
Params2.67B
Context32K
Best fitQ8_0 4.0 GB
GGUF
GL
GLM 4.7 Flash Name NSFW
GLM · mradermacher
Params30B
Context198K
Q4_K_M18.1 GB
GGUF
AS
astrollama 3 8b chat aic i1
Astrollama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
IQ
IQuest Coder V1 40B Instruct
Iquest · mradermacher
Params39.8B
Context128K
Q4_K_M24.0 GB
Coding
SK
Skywork OR1 Math 7B
Skywork · bartowski
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
GGUF
IN
Internlm2 Math Base 7b
InternLM · mradermacher
Params7.74B
Context8K
Best fitIQ2_XS 4.0 GB
GGUF
PR
PreciseCoder 4B
Precisecoder · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
Coding
QW
Qwen2 Math 1.5B Instruct
Qwen · legraphista
Params1.54B
Context4K
Best fitBF16 3.8 GB
GGUF
PY
Pythia 160m
Pythia · mradermacher
Params162M
Context2K
Best fitIQ1_S 0.9 GB
GGUF
IN
Index Nailong 2B
Index · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
GGUF
GP
Gpt Oss Sanguine 20b v1
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
CodingReasoningRoleplay
QW
Qwen3 Reranker 0.6B
Qwen · mradermacher
Params596M
Context40K
Best fitF16 2.6 GB
Embedding
QW
Qwen3 Esper3 Reasoning CODER Instruct 6B Brainstorm20x
Qwen · mradermacher
Params5.94B
Context40K
Best fitIQ1_M 3.9 GB
CodingReasoning

Other memory sizes

Keep browsing