GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
GP
Gpt Oss Sanguine 20b v1
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
CodingReasoningRoleplay
OL
Olmo-3-1025-7B
OLMo · Mungert
Params7.3B
Context64K
Q4_K_M4.7 GB
GGUF
QW
Qwen3 Reranker 0.6B
Qwen · mradermacher
Params596M
Context40K
Best fitF16 2.6 GB
Embedding
QW
Qwen3 Esper3 Reasoning CODER Instruct 6B Brainstorm20x
Qwen · mradermacher
Params5.94B
Context40K
Best fitQ4_K_M 6.0 GB
CodingReasoning
GE
Winnow E2B
Gemma · EldanRing
Params77M
Context128K
Best fitQ8_0 5.5 GB
Vision
CH
Chaotic Order V4
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
NE
Nemotron-3.5-Bolt-E4B-Heretic
Nemotron · grumd0
Params7.46B
Context64K
Best fitQ2_K_XL 5.6 GB
ReasoningTool useVision
MI
Mistral 7B Cymraeg Welsh v2
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_S 5.9 GB
GGUF
HO
K2 Horizon 3.7B Claude Opus Reasoning Distilled
Horizon · mradermacher
Params5.06B
Context512K
Best fitQ6_K 5.9 GB
ReasoningTool use
AL
AliceAI Foundation 80B A3B NOESIS BF16
Aliceai · AMAImedia
Params79.6B
Context256K
Best fitF16 4.6 GB
Coding
GO
Goetia 24B v1.4 Heretic
Goetia · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
Roleplay
TO
ToolSpace 35B A3B
Toolspace · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool use
QW
qwen3 4b heretic zimage
Qwen · Lockout
Params4B
Context40K
Smallest4.3 GB
GGUF
GE
Gemma 2 2b It
Gemma · l3utterfly
Params2.61B
Context8K
Best fitQ8_0 4.2 GB
GGUF
QW
Qwen2.5 3B
Qwen · mradermacher
Params3.09B
Context32K
Best fitQ6_K 3.3 GB
GGUF
QW
ColdBrew Qwen3.5 RP_Base CharThink
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_S 6.0 GB
ReasoningRoleplayTool use
MI
YuisekinAIEvol Mistral 7B ja math v0.1.1
Mistral · mmnga
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
IN
InternVL3_5 4B
Internvl · bartowski
Params4.41B
Context40K
Best fitQ6_K_L 5.5 GB
Vision
SN
Snowpiercer 15B v4
Snowpiercer · bartowski
Params15B
Context64K
Q4_K_M9.1 GB
GGUF
DE
Deepseek R1 Distill NSFW RPv1
DeepSeek · mradermacher
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
Reasoning
WO
WorldSim Opus 3.6 35B A3B
Worldsim · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
ReasoningRoleplayVision
EX
EXAONE 3.5 7.8B Instruct
Exaone · ThomasBaruzier
Params7.8B
Context32K
Best fitIQ4_XS 5.9 GB
GGUF
QW
ThinCtx Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
GGUF
D3
d3 i1
mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
Vision
OP
weyaxi_openchat-3.5-1210-seraph-slerp
Openchat · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
LL
Llama 3 Meerkat 8b v1.0
Llama · RichardErkhov
Params8B
Context8K
Best fitQ3_K_L 5.9 GB
GGUF
LF
LFM2.5 2.6B Uncensored Trial65
Lfm · SC117
Params2.7B
Context125K
Best fitQ8_0 5.4 GB
GGUF
LI
Lily-Cybersecurity-7B-v0.2
Lily · RichardErkhov
Params7.24B
Context32K
Best fitQ4_K 5.9 GB
GGUF
LF
LFM3-Omni-2B
Lfm · AtomicChat
Params1.73B
Context125K
Best fitQ8_0 4.0 GB
ReasoningTool useVision
MI
MiMo V2.6 Flash REAP50
Mimo · patrickbdevaney
Params157B
Context1024K
Best fitBF16 6.0 GB
ReasoningTool useVision

Other memory sizes

Keep browsing