GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
TI
jeff31415_tinyllama-1.1b-1t-openorca
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
NE
Daredevil-8B-abliterated-dpomix
Neuraldaredevil · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
BO
Borealis Open 270m
Borealis · mradermacher
Params268M
Context32K
Best fitIQ1_S 0.9 GB
B3
B3 27B i1
mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
FA
Falcon-H1-3B-Instruct
Falcon · unsloth
Params3.15B
Context128K
Best fitQ6_K_XL 3.6 GB
QW
Qwen3.6-35B-A3B-MTP
Qwen · Ninnix96
Params34.7B
Context256K
Best fitQ8_0 2.8 GB
AM
amalia-desenrasca-9b
Amalia · mradermacher
Params9.15B
Context32K
Q4_K_M5.6 GB
SP
Spikingkiki 122b A10B
Spikingkiki · mradermacher
Params122B
Context256K
Q4_K_M74.2 GB
DA
Darwin 27B RSI
Darwin · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
AU
Audar TTS V1 Flash
Audar · audarai
Params553M
Context32K
Best fitQ8_0 1.2 GB
LL
Llama3 8b Cpt Sea Lionv2 Instruct
Llama · RichardErkhov
Params8B
Context8K
Best fitIQ4_NL 3.3 GB
QW
Qwen3 VL 2B Thinking
Qwen · bartowski
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
QW
Qwen2.5 VL 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
ME
MedMO 8B Next
Medmo · mradermacher
Params8.19B
Context256K
Best fitIQ1_M 4.0 GB
HE
HelpingAI 3B hindi i1
Helpingai · mradermacher
Params2.8B
Context4K
Best fitIQ2_XXS 4.0 GB
QW
Qwen2.5-Vl-7B-Instruct
Qwen · mnsims1024
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
MI
Nexus IKM Mistral Instruct v0.2 7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
YO
Youtu-VL-4B-Instruct
Youtu · Mungert
Params4.9B
Context32K
Best fitQ3_K_M 4.0 GB
MA
Marco o1
Marco · bartowski
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · DuoNeural
Params7.6B
Context32K
Best fitIQ2_M 3.8 GB
GE
Gemma 3 4b It Uncensored
Gemma · mradermacher
Params3.88B
Context128K
Best fitQ4_K_S 4.0 GB
HA
Hathor_Unstable-L3-8B-v0.3
Hathor · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
BI
BioMistral 7B
Biomistral · BioMistral
Params7.24B
Context32K
Best fitQ3_K_L 2.6 GB
QW
Qwen3.5 35B A3B Sceptic v3.4
Qwen · kataguru
Params35.5B
Context256K
Best fitQ8_0 3.2 GB
QW
Swift Qwen3.8 27B Uncensored MTP Terse Coder
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
ER
Erebus RP 12B Instruct 2608 v1
Erebus · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
BO
Borealis 4b
Borealis · mradermacher
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
QW
Qwen3.5 27B RpRMax v1
Qwen · bartowski
Params27B
Context256K
Q4_K_M17.5 GB
EV
EverythingLM 13b V2 16K
Everythinglm · TheBloke
Params13B
Context—
Best fitQ5_0 0.5 GB
GE
Gemma 4 E4B It The DECKARD Claude Opus Expresso Universe HERETIC UNCENSORED Thinking
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB