GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
TI
jeff31415_tinyllama-1.1b-1t-openorca
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
Coding
NE
Daredevil-8B-abliterated-dpomix
Neuraldaredevil · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
BO
Borealis Open 270m
Borealis · mradermacher
Params268M
Context32K
Best fitIQ1_S 0.9 GB
GGUF
B3
B3 27B i1
mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
ReasoningTool use
FA
Falcon-H1-3B-Instruct
Falcon · unsloth
Params3.15B
Context128K
Best fitQ6_K_XL 3.6 GB
GGUF
QW
Qwen3.6-35B-A3B-MTP
Qwen · Ninnix96
Params34.7B
Context256K
Best fitQ8_0 2.8 GB
ReasoningTool use
AM
amalia-desenrasca-9b
Amalia · mradermacher
Params9.15B
Context32K
Q4_K_M5.6 GB
Tool use
SP
Spikingkiki 122b A10B
Spikingkiki · mradermacher
Params122B
Context256K
Q4_K_M74.2 GB
GGUF
DA
Darwin 27B RSI
Darwin · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
Reasoning
AU
Audar TTS V1 Flash
Audar · audarai
Params553M
Context32K
Best fitQ8_0 1.2 GB
GGUF
LL
Llama3 8b Cpt Sea Lionv2 Instruct
Llama · RichardErkhov
Params8B
Context8K
Best fitIQ4_NL 3.3 GB
GGUF
QW
Qwen3 VL 2B Thinking
Qwen · bartowski
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
ReasoningVision
QW
Qwen2.5 VL 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
CodingVision
ME
MedMO 8B Next
Medmo · mradermacher
Params8.19B
Context256K
Best fitIQ1_M 4.0 GB
Reasoning
HE
HelpingAI 3B hindi i1
Helpingai · mradermacher
Params2.8B
Context4K
Best fitIQ2_XXS 4.0 GB
GGUF
QW
Qwen2.5-Vl-7B-Instruct
Qwen · mnsims1024
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
Vision
MI
Nexus IKM Mistral Instruct v0.2 7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
YO
Youtu-VL-4B-Instruct
Youtu · Mungert
Params4.9B
Context32K
Best fitQ3_K_M 4.0 GB
Vision
MA
Marco o1
Marco · bartowski
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct
Qwen · DuoNeural
Params7.6B
Context32K
Best fitIQ2_M 3.8 GB
Coding
GE
Gemma 3 4b It Uncensored
Gemma · mradermacher
Params3.88B
Context128K
Best fitQ4_K_S 4.0 GB
Vision
HA
Hathor_Unstable-L3-8B-v0.3
Hathor · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
BI
BioMistral 7B
Biomistral · BioMistral
Params7.24B
Context32K
Best fitQ3_K_L 2.6 GB
GGUF
QW
Qwen3.5 35B A3B Sceptic v3.4
Qwen · kataguru
Params35.5B
Context256K
Best fitQ8_0 3.2 GB
ReasoningTool useVision
QW
Swift Qwen3.8 27B Uncensored MTP Terse Coder
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
CodingReasoning
ER
Erebus RP 12B Instruct 2608 v1
Erebus · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
Roleplay
BO
Borealis 4b
Borealis · mradermacher
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
GGUF
QW
Qwen3.5 27B RpRMax v1
Qwen · bartowski
Params27B
Context256K
Q4_K_M17.5 GB
Vision
EV
EverythingLM 13b V2 16K
Everythinglm · TheBloke
Params13B
Context—
Best fitQ5_0 0.5 GB
GGUF
GE
Gemma 4 E4B It The DECKARD Claude Opus Expresso Universe HERETIC UNCENSORED Thinking
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
CodingReasoningRoleplay

Other memory sizes

Keep browsing