GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
ZE
Zephyr 7b Beta Abliterated
Zephyr · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
QW
Qwen35B Agent R2 Abliterated
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
KL
Klyrune 2B General
Klyrune · mradermacher
Params2.52B
Context128K
Best fitQ6_K 2.9 GB
RO
RolmOCR
Rolmocr · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
BO
Borealis Open 1b
Borealis · mradermacher
Params1B
Context32K
Best fitQ6_K 1.7 GB
LL
alfredplpl Llama 3 8B Instruct Ja
Llama · mmnga
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
QW
VellumMini 0.1 Qwen3 14B
Qwen · bartowski
Params14.8B
Context40K
Q4_K_M9.0 GB
QW
ThinkingCap Qwen3.6 27B Heretic
Qwen · egibson
Params27B
Context256K
Q4_K_M16.5 GB
QW
Qwen3.5 122B A10B Abliterated
Qwen · mradermacher
Params122B
Context256K
Q4_K_M74.2 GB
LF
Kulyk En Uk
Lfm · Yehor
Params354M
Context125K
Best fitQ8_0 1.4 GB
GE
TheDrummer Gemmasutra Small 4B v1
Gemma · bartowski
Params3.86B
Context8K
Best fitQ3_K_M 4.0 GB
CO
Yi Coder 9B
Coder · mradermacher
Params8.83B
Context128K
Best fitIQ2_XS 4.0 GB
NE
Orchestrator 8B
Nemotron · bartowski
Params8.19B
Context40K
Q4_K_M5.0 GB
GE
Gemma 3 1b It
Gemma · gguf-org
Params1B
Context32K
Best fitBF16 2.7 GB
MO
MOSS Image Text Encoder Heretic
Moss · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
GE
Gemma 3 12b It Vl HighIQ Polaris Heretic Uncensored Thinking
Gemma · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
DE
deepseek-coder-1.3B-kexer
DeepSeek · lmstudio-community
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
KA
Kairos
Kairos · mradermacher
Params8.2B
Context32K
Best fitIQ1_M 4.0 GB
CO
COBOL Coder 7B Instruct i1
Cobol · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
LL
Llama3.1 Nemo Thinker
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
OP
Openhands Lm 1.5b v0.1
Openhands · bartowski
Params1.54B
Context32K
Best fitBF16 3.8 GB
GR
Grug 35B A3B
Grug · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
ST
Starcoder2 3b
StarCoder · mradermacher
Params3.03B
Context16K
Best fitQ8_0 4.0 GB
FU
Functiongemma 270m It
Functiongemma · bartowski
Params268M
Context32K
Best fitBF16 1.2 GB
BO
Borealis Open 4b
Borealis · mradermacher
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
QW
Qwen2.5 0.5B Instruct
Qwen · mradermacher
Params494M
Context32K
Best fitQ6_K 1.1 GB
VE
Veloxy SLM 3B
Veloxy · mradermacher
Params3.09B
Context32K
Best fitQ6_K 3.3 GB
MI
MiMo Ornith 9B AGSI Abliterated HQ
Mimo · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
BE
Behemoth T1 123B
Behemoth · mradermacher
Params123B
Context128K
Q4_K_M73.2 GB
QW
Qwen2.5 3B Instruct heretic
Qwen · saidutta69
Params3.09B
Context32K
Best fitQ6_K 3.3 GB