GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
DE
WhiteRabbitNeo V3 7B i1
Deephat · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
NA
Nanbeige4.3 33B Base
Nanbeige · mradermacher
Params33.1B
Context128K
Q4_K_M20.1 GB
LL
Turkish Llama 8b Instruct v0.1
Llama · ytu-ce-cosmos
Params8B
Context8K
Best fitIQ2_M 2.2 GB
DA
danube2-1.8b-CodeFeedback
Danube · mradermacher
Params1.83B
Context8K
Best fitQ8_0 3.0 GB
QW
Git Commit Message Splitter Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
QW
Qwen2.5 Coder 1.5B
Qwen · QuantFactory
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
LL
llama nemotron embed 1b v2
Llama · sabafallah
Params1.24B
Context128K
Best fitQ8_0 2.1 GB
QW
Qwen3 DND Jan v1 256k Ctx Brainstorm40x 8B
Qwen · mradermacher
Params7.66B
Context256K
Q4_K_M4.7 GB
IC
IceEspressoRPv1 7b i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
QW
Replete LLM V2.5 Qwen 7b
Qwen · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GE
Fallen Gemma3 4B v1
Gemma · bartowski
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
SO
SOCAM V1
Socam · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
AG
Agents A1 4B
Agents · bartowski
Params4.21B
Context256K
Best fitQ3_K_M 4.0 GB
LF
LFM2.5 2.6B Tema_Q Abliterated
Lfm · mradermacher
Params2.7B
Context128K
Best fitQ3_K_L 4.0 GB
AA
aac pictogram retriever ptbr
Aac · Papuguinho
Params155M
Context2K
Best fitF16 1.0 GB
VI
Video ORA 9B
Video · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
LL
Llama 3.3 70B Instruct Heretic
Llama · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
GR
granite 3.1 2b instruct
Granite · QuantFactory
Params2.63B
Context128K
Best fitQ8_0 4.0 GB
DE
deepseek-coder-1.3b-instruct
DeepSeek · duyntnet
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
TI
TinySolar-248m-4k-py
Tinysolar · RichardErkhov
Params248M
Context32K
Best fitQ8_0 0.9 GB
YI
Yi-1.5-9B
Yi · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
MA
Mamba2 130m Hf
Mamba · rpatel622
Params168M
Context1024K
Best fitQ8_0 0.7 GB
QW
Qwen3.8 27B Uncensored FP8
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
QW
Qwen3.8 27B Continuum
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
QW
Qwen3.6 Fallen Fabulist 35B A3B Heretic
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
QW
Qwen3 Esper3 Reasoning Instruct 6B Brainstorm20x Enhanced E32
Qwen · mradermacher
Params5.94B
Context40K
Best fitIQ1_M 3.9 GB
LL
Llama 3.1 Nemotron Nano 4B v1.1
Llama · bartowski
Params4.51B
Context128K
Best fitQ3_K_L 4.0 GB
GE
Gemma 2 2b It
Gemma · BafS
Params2.61B
Context8K
Best fitQ4_K_M 3.1 GB