GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5 4B EmperoAI Qwen3.8 Distill Heretic Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
ReasoningTool use
LL
LlamaScientific 8B
Llama · mradermacher
Params8.03B
Context128K
Best fitIQ2_XXS 4.0 GB
Tool use
JI
Jina Reranker v3.5
Jina · jinaai
Params596M
Context128K
Best fitBF16 2.6 GB
Embedding
QW
Qwen3.5 4B DFlash
Qwen · magnitudedev
Params634M
Context40K
Best fitQ8_0 1.4 GB
GGUF
MI
Ministral-3-3B-Reasoning-2512
Mistral · Mungert
Params3.43B
Context256K
Best fitQ5_K_M 4.0 GB
Reasoning
QW
Agrillm Qwen3 30B A3B
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
GGUF
QW
Qwen2.5 VL 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
CodingVision
SM
KrynexAI 1.4
SmolLM · KrynexLabs
Params135M
Context8K
Best fitF16 1.0 GB
GGUF
QW
Co Rewarding I Qwen3 8B Base MATH
Qwen · mradermacher
Params8.19B
Context32K
Best fitIQ1_M 4.0 GB
GGUF
SO
Socio Foundation 8B
Socio · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
GGUF
ME
Hy Memory 30B A3B
Memory · vinnipukh
Params30B
Context256K
Q4_K_M18.2 GB
GGUF
LF
LFM2 VL 450M
Lfm · LiquidAI
Params354M
Context125K
Best fitF16 1.7 GB
Vision
VA
Valkyrie 49B v2.1
Valkyrie · bartowski
Params49.9B
Context128K
Q4_K_M30.2 GB
GGUF
GP
Gpt Oss 120b
gpt-oss · bartowski
Params117B
Context128K
Q4_K_M62.8 GB
GGUF
QW
Qwen3 Coder 30B A3B Instruct Westernized
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
Coding
CL
CLM v0.1 8B
Clm · czl
Params8.19B
Context40K
Q4_K_M5.0 GB
Embedding
SH
Shieldgemma 2b
Shieldgemma · legraphista
Params2.61B
Context8K
Best fitQ6_K 3.5 GB
GGUF
QW
Qwen3.6 27B ABLITERATED UNCENSORED PHILADELPHIA CLASS
Qwen · mradermacher
Params26.9B
Context256K
Q4_K_M16.5 GB
ReasoningTool useVision
QW
dolphin-2.9.3-qwen2-1.5b
Qwen · crusoeai
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
AM
Amaretto 8B
Amaretto · mradermacher
Params8.49B
Context256K
Best fitIQ1_M 3.9 GB
Tool use
HA
Haru Kanana 8b
Haru · mradermacher
Params8B
Context32K
Best fitIQ2_XXS 4.0 GB
Roleplay
YI
Yi-1.5-9B-Chat-16K
Yi · mradermacher
Params8.83B
Context16K
Best fitIQ2_XS 4.0 GB
GGUF
SM
nisten/Biggie-SmoLlm-0.15B-Base
SmolLM · nisten
Params152M
Context2K
Best fitBF16 1.1 GB
GGUF
QW
QiMing Holos Plus Qwen3 14B
Qwen · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
GGUF
ST
L3-8B-Stheno-v3.2
Stheno · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
QW
Jan-v2-VL-high
Qwen · Mungert
Params8.2B
Context256K
Q4_K_M5.4 GB
ReasoningVision
QW
Qwen3.6 35B A3B Uncensored Aggressive
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
Reasoning
QW
Qwen3 VL 32B Instruct Ultra Uncensored Heretic
Qwen · mradermacher
Params32.8B
Context256K
Q4_K_M19.8 GB
Vision
HO
Holo4 27B Abliterix
Holo · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
Vision
QW
Qwen2.5 Coder 0.5B Instruct Abliterated
Qwen · bartowski
Params494M
Context32K
Best fitF16 1.6 GB
Coding

Other memory sizes

Keep browsing