GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
ST
Storm-7B
Storm · mradermacher
Params7.24B
Context8K
Best fitIQ2_S 3.9 GB
AC
AceInstruct 7B
Aceinstruct · bartowski
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
PH
Phind 70B
Phind · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
KI
Kimi K2.5
Kimi · bartowski
Params1030B
Context256K
Q4_K_M580.4 GB
LL
llama-3-8b-samantha
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
CO
CodeQwen1.5-7B
Codeqwen · bartowski
Params7.25B
Context64K
Best fitIQ2_M 4.0 GB
JI
Jina Embeddings v5 Text Nano Clustering
Jina · jinaai
Params212M
Context8K
Best fitF16 1.2 GB
BL
BlackMagic-7B
Blackmagic · mradermacher
Params7.24B
Context8K
Best fitIQ2_S 3.9 GB
PY
Pythia 1b
Pythia · mradermacher
Params1.01B
Context2K
Best fitF16 3.6 GB
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · ggml-org
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
QW
Qwen2.5 Furry 7B
Qwen · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
QW
Qwen3.8 27B DSpark
Qwen · Anbeeld
Params1.86B
Context256K
Best fitQ8_0 2.7 GB
QW
Haru Qwen3 4b
Qwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
MI
ANIMA Cognitive Mistral v1 i1
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GL
LFM2 12B A1B GLM 4.7 Thinking Quantum IQ1C P TR1 S2
GLM · mradermacher
Params11.3B
Context125K
Q4_K_M6.8 GB
ST
Starcoder2 7b
StarCoder · QuantFactory
Params7.17B
Context16K
Best fitQ2_K 3.9 GB
ME
Merlin Agent
Merlin · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
OR
Orpheus 3b 0.1 Pretrained GGUF
Orpheus · HorizonNexusAI
Params3.3B
Context128K
Best fitQ5_K_M 3.8 GB
CO
Codeqwen3 14b
Codeqwen · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
QW
OPSD Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
OP
OpenELM-270M-Instruct
Openelm · RichardErkhov
Params272M
Context2K
Best fitQ6_K 0.8 GB
RA
RakutenAI 7B
Rakutenai · mradermacher
Params7.37B
Context32K
Best fitIQ2_S 4.0 GB
IN
Internvl3-8B
Internvl · unsloth
Params7.61B
Context32K
Best fitQ2_K 4.0 GB
TI
TigerCoder 12B
Tigercoder · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
PR
PromptEnhancer 32B
Promptenhancer · mradermacher
Params32.8B
Context125K
Q4_K_M19.9 GB
QW
X Coder SFT Qwen3 8B
Qwen · mradermacher
Params8.19B
Context32K
Best fitIQ1_M 4.0 GB
QW
Qwen3.5 0.8B Heretic
Qwen · saidutta69
Params752M
Context256K
Best fitF16 2.4 GB
LL
Llama-3-Mopeyfied-Psychology-v2
Llama · mradermacher
Params8.03B
Context8K
Best fitIQ2_XXS 4.0 GB
LL
llama3-8B-DarkIdol-2.1-Uncensored-1048K
Llama · mradermacher
Params8B
Context1024K
Best fitIQ2_XXS 4.0 GB
QW
qwen2.5 coder 32b combined verilog c i1
Qwen · mradermacher
Params32.8B
Context32K
Q4_K_M19.8 GB