GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
GP
Gpt Oss Sanguine 20b v1
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
OL
Olmo-3-1025-7B
OLMo · Mungert
Params7.3B
Context64K
Q4_K_M4.7 GB
QW
Qwen3 Reranker 0.6B
Qwen · mradermacher
Params596M
Context40K
Best fitF16 2.6 GB
QW
Qwen3 Esper3 Reasoning CODER Instruct 6B Brainstorm20x
Qwen · mradermacher
Params5.94B
Context40K
Best fitQ4_K_M 6.0 GB
GE
Winnow E2B
Gemma · EldanRing
Params77M
Context128K
Best fitQ8_0 5.5 GB
CH
Chaotic Order V4
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
NE
Nemotron-3.5-Bolt-E4B-Heretic
Nemotron · grumd0
Params7.46B
Context64K
Best fitQ2_K_XL 5.6 GB
MI
Mistral 7B Cymraeg Welsh v2
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_S 5.9 GB
HO
K2 Horizon 3.7B Claude Opus Reasoning Distilled
Horizon · mradermacher
Params5.06B
Context512K
Best fitQ6_K 5.9 GB
AL
AliceAI Foundation 80B A3B NOESIS BF16
Aliceai · AMAImedia
Params79.6B
Context256K
Best fitF16 4.6 GB
GO
Goetia 24B v1.4 Heretic
Goetia · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
TO
ToolSpace 35B A3B
Toolspace · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
qwen3 4b heretic zimage
Qwen · Lockout
Params4B
Context40K
Smallest4.3 GB
GE
Gemma 2 2b It
Gemma · l3utterfly
Params2.61B
Context8K
Best fitQ8_0 4.2 GB
QW
Qwen2.5 3B
Qwen · mradermacher
Params3.09B
Context32K
Best fitQ6_K 3.3 GB
QW
ColdBrew Qwen3.5 RP_Base CharThink
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_S 6.0 GB
MI
YuisekinAIEvol Mistral 7B ja math v0.1.1
Mistral · mmnga
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
IN
InternVL3_5 4B
Internvl · bartowski
Params4.41B
Context40K
Best fitQ6_K_L 5.5 GB
SN
Snowpiercer 15B v4
Snowpiercer · bartowski
Params15B
Context64K
Q4_K_M9.1 GB
DE
Deepseek R1 Distill NSFW RPv1
DeepSeek · mradermacher
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
WO
WorldSim Opus 3.6 35B A3B
Worldsim · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
EX
EXAONE 3.5 7.8B Instruct
Exaone · ThomasBaruzier
Params7.8B
Context32K
Best fitIQ4_XS 5.9 GB
QW
ThinCtx Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
D3
d3 i1
mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
OP
weyaxi_openchat-3.5-1210-seraph-slerp
Openchat · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
LL
Llama 3 Meerkat 8b v1.0
Llama · RichardErkhov
Params8B
Context8K
Best fitQ3_K_L 5.9 GB
LF
LFM2.5 2.6B Uncensored Trial65
Lfm · SC117
Params2.7B
Context125K
Best fitQ8_0 5.4 GB
LI
Lily-Cybersecurity-7B-v0.2
Lily · RichardErkhov
Params7.24B
Context32K
Best fitQ4_K 5.9 GB
LF
LFM3-Omni-2B
Lfm · AtomicChat
Params1.73B
Context125K
Best fitQ8_0 4.0 GB
MI
MiMo V2.6 Flash REAP50
Mimo · patrickbdevaney
Params157B
Context1024K
Best fitBF16 6.0 GB