GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
online-dpo-qwen2-4
Qwen · RichardErkhov
Params494M
Context32K
Best fitQ6_K 1.1 GB
GGUF
QW
hamo score 0.6b
Qwen · HamoAI
Params596M
Context40K
Smallest0.6 GB
GGUF
HE
NousResearch.Hermes 4 405B
Hermes · DevQuasar-13
Params405B
Context128K
Q4_K_M243.1 GB
Tool use
GE
Gemma-3N-E4B-It
Gemma · unsloth
Params6.9B
Context32K
Best fitQ5_K_S 6.0 GB
Vision
QW
ColdBrew Qwen3.5 CharThink Base
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_S 6.0 GB
ReasoningTool use
LL
Vikhr-Llama-3.2-1B-instruct
Llama · Vikhrmodels
Params1.24B
Context128K
Best fitF16 3.2 GB
GGUF
QW
RuntimeLabs Kayak Qwen3 VL 4B
Qwen · mradermacher
Params4B
Context256K
Best fitQ8_0 6.0 GB
Vision
SA
Salamandra 7b Instruct
Salamandra · RichardErkhov
Params7.77B
Context8K
Best fitQ3_K_L 5.9 GB
GGUF
LF
LFM2 2.6B Exp
Lfm · LiquidAI
Params2.7B
Context125K
Best fitQ8_0 5.4 GB
GGUF
GT
gte-large
Gte · opaninakuffo
Params334M
Context512
Smallest0.7 GB
GGUF
GE
Octopus-v2
Gemma · NexaAI
Params2.51B
Context8K
Best fitF16 5.7 GB
GGUF
QW
Qwen3 Embedding 0.6B
Qwen · PeterAM4
Params596M
Context32K
Best fitBF16 2.6 GB
Embedding
CO
ContextualKunoichi KTO 7B i1
Contextualkunoichi · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
QW
Qwen2 0.5B NuminaMath TIR v1
Qwen · mradermacher
Params494M
Context32K
Best fitF16 1.6 GB
GGUF
LF
LFM2.5 8B A1B Uncensored
Lfm · zaakirio
Params8.47B
Context125K
Best fitQ3_K_S 5.9 GB
GGUF
ST
stabilityai japanese stablelm base beta 70b
Stabilityai · RichardErkhov
Params1.6B
Context—
Best fitQ3_K_S 5.3 GB
GGUF
DE
DeepHat V1 7B imatrix
Deephat · liodon-ai
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
Coding
ZE
zephyr 7b hh dpo i1
Zephyr · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
ST
starcoder2-7b
StarCoder · second-state
Params7.17B
Context16K
Best fitQ5_0 6.0 GB
Coding
AP
Apriel 1.5 15b Thinker
Apriel · mradermacher
Params14.4B
Context256K
Best fitIQ1_M 5.8 GB
ReasoningVision
QW
Qwen3.5 2B
Qwen · SoAIHQ
Params1.94B
Context256K
Best fitQ8_0 3.0 GB
ReasoningTool useVision
LL
lightblue suzume llama 3 8B japanese
Llama · mmnga
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
GGUF
QW
Qwen2.5 Wolverine CODER 11B V2 128k Ctx
Qwen · mradermacher
Params10.9B
Context128K
Best fitIQ3_XS 5.9 GB
Coding
QW
dnotitia_Smoothie Qwen3 0.6B
Qwen · Alcoft
Params596M
Context40K
Q4_K_M0.4 GB
ReasoningTool use
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · unsloth
Params1.54B
Context32K
Best fitF16 3.8 GB
Coding
GR
Granite-4.0-Micro
Granite · unsloth
Params3.4B
Context128K
Best fitQ4_K_M 6.0 GB
GGUF
NE
Nemotron 3.5 Bolt E4B Heretic
Nemotron · mradermacher
Params7.52B
Context64K
Q4_K_M5.3 GB
ReasoningTool use
DI
DiscoveryRSI 27B
Discoveryrsi · bartowski
Params27.3B
Context256K
Q4_K_M17.4 GB
ReasoningTool useVision
LL
OneLLM Doey V1 Llama 3.2 1B It
Llama · DoeyLLM
Params1.24B
Context128K
Best fitQ8_0 2.1 GB
GGUF
GE
Gemma 3 4b It Abliterated
Gemma · mlabonne
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
Vision

Other memory sizes

Keep browsing