GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
NE
Nemotron-3.5-Bolt-E4B-Heretic
Nemotron · grumd0
Params7.46B
Context64K
Best fitQ5_K_XL 7.7 GB
MI
Mistral 7B Cymraeg Welsh v2
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.7 GB
HO
K2 Horizon 3.7B Claude Opus Reasoning Distilled
Horizon · mradermacher
Params5.06B
Context512K
Best fitQ6_K 5.9 GB
AL
AliceAI Foundation 80B A3B NOESIS BF16
Aliceai · AMAImedia
Params79.6B
Context256K
Best fitF16 4.6 GB
DE
Deepseek Coder 6.7b Instruct
DeepSeek · mradermacher
Params6.74B
Context16K
Best fitIQ3_M 7.9 GB
GO
Goetia 24B v1.4 Heretic
Goetia · mradermacher
Params23.6B
Context128K
Best fitIQ1_M 7.6 GB
TO
ToolSpace 35B A3B
Toolspace · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
qwen3 4b heretic zimage
Qwen · Lockout
Params4B
Context40K
Smallest4.3 GB
MI
Mistral Nemo Instruct 2407
Mistral · QuantFactory
Params12.2B
Context1000K
Best fitQ3_K_M 7.9 GB
GE
Gemma 2 2b It
Gemma · l3utterfly
Params2.61B
Context8K
Best fitF16 6.6 GB
QW
Qwen2.5 3B
Qwen · mradermacher
Params3.09B
Context32K
Best fitQ6_K 3.3 GB
QW
ColdBrew Qwen3.5 RP_Base CharThink
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ5_K_S 7.9 GB
MI
YuisekinAIEvol Mistral 7B ja math v0.1.1
Mistral · mmnga
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
IN
InternVL3_5 4B
Internvl · bartowski
Params4.41B
Context40K
Best fitQ8_0 6.4 GB
SN
Snowpiercer 15B v4
Snowpiercer · bartowski
Params15B
Context64K
Best fitQ2_K 8.0 GB
DE
Deepseek R1 Distill NSFW RPv1
DeepSeek · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
WO
WorldSim Opus 3.6 35B A3B
Worldsim · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
EX
EXAONE 3.5 7.8B Instruct
Exaone · ThomasBaruzier
Params7.8B
Context32K
Best fitQ6_K 8.0 GB
QW
ThinCtx Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
D3
d3 i1
mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
OP
weyaxi_openchat-3.5-1210-seraph-slerp
Openchat · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
LL
Llama 3 Meerkat 8b v1.0
Llama · RichardErkhov
Params8B
Context8K
Best fitQ5_1 7.6 GB
GE
Gemma 3 12b It Qat
Gemma · bartowski
Params11.8B
Context128K
Best fitIQ2_M 8.0 GB
LF
LFM2.5 2.6B Uncensored Trial65
Lfm · SC117
Params2.7B
Context125K
Best fitBF16 7.9 GB
QW
Qwen2.5 Coder 14B Instruct
Qwen · gaianet
Params14.8B
Context32K
Best fitQ2_K 7.9 GB
LI
Lily-Cybersecurity-7B-v0.2
Lily · RichardErkhov
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
AM
llm360_amberchat
Amberchat · TheBloke
Params6.74B
Context2K
Best fitQ3_K_S 7.7 GB
LF
LFM3-Omni-2B
Lfm · AtomicChat
Params1.73B
Context125K
Best fitQ8_0 4.0 GB
MI
MiMo V2.6 Flash REAP50
Mimo · patrickbdevaney
Params157B
Context1024K
Best fitBF16 6.0 GB
QW
Roblox Coder v2
Qwen · preferedev
Params4.33B
Context256K
Best fitQ8_0 6.2 GB