GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
HE
0 hero Matter 0.2 7B
Hero · RichardErkhov
Params7.24B
Context32K
Best fitQ4_K 5.9 GB
LL
llama3.1 airoboros3.2 QDT 8B
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
IN
InternVL3_5 2B
Internvl · bartowski
Params2.03B
Context40K
Best fitBF16 5.5 GB
LL
LIama 3.2 test
Llama · bhanuripranathi
Params3.21B
Context128K
Best fitQ4_K_M 3.5 GB
QW
Qwen3.8 Flash Next
Qwen · AesSedai
Params180B
Context256K
Q4_K_M116.1 GB
AR
AREX Turbo
Arex · bartowski
Params4.2B
Context256K
Best fitQ6_K_L 5.4 GB
BO
Borealis Open 27b
Borealis · mradermacher
Params27B
Context128K
Q4_K_M16.5 GB
QW
Qwen3.8 27B Uncensored
Qwen · shsunshining
Params27.3B
Context256K
Best fitQ8_0 5.8 GB
QW
Huihui Qwen3.6 35B A3B Claude 4.7 Opus Abliterated
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen2.5 Coder 1.5B
Qwen · ggml-org
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
MI
MegaBeam Mistral 7B 512k
Mistral · bartowski
Params7.24B
Context512K
Best fitQ4_K_L 6.0 GB
HY
Hy MT2 1.8B 1.25Bit
tencent
Params1.79B
Context256K
Smallest0.5 GB
JE
Jebadiah 9b v2
Jebadiah · bartowski
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
TI
TinyLlama 1.1B Chat v0.6
Tinyllama · TinyLlama
Params1.1B
Context2K
Best fitQ4_0 1.3 GB
MI
ehartford_dolphin-2.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
DE
DeepSeek R1 Distill Llama 8B
DeepSeek · mradermacher
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
GE
Gemma 3 270m Qat Q4_0 Unquantized
Gemma · ggml-org
Params268M
Context32K
Best fitQ4_0 0.9 GB
QW
Kataguru Sceptic 35B A3B v2.6
Qwen · kataguru
Params35.2B
Context256K
Best fitQ8_0 3.2 GB
LL
Llm Jp 4 8b Instruct
Llm · mmnga-o
Params8.59B
Context64K
Best fitQ3_K_M 5.9 GB
QW
Qwen2.5-Omni-3B
Qwen · unsloth
Params3.4B
Context32K
Best fitQ8_K_XL 5.4 GB
NE
Nemotron 3.5 Bolt E4B GSQ RCO
Nemotron · abenzerps
Params7.52B
Context128K
Best fitIQ4_XS 5.5 GB
CA
CAT Thinking 8B
Cat · mmnga-o
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
IN
InternLM2
InternLM · legraphista
Params7.74B
Context8K
Best fitIQ4_NL 6.0 GB
NE
NVIDIA Nemotron 3.5 Bolt E4B Pollard
Nemotron · PollardWeights
Params7.52B
Context128K
Q4_K_M0.2 GB
QW
Qwen2.5 3B Instruct
Qwen · lmstudio-community
Params3.09B
Context32K
Best fitQ8_0 4.1 GB
QW
Huihui Qwen3 4B Abliterated v2
Qwen · mradermacher
Params4B
Context40K
Best fitQ6_K 5.0 GB
QW
Qwen3 4B Thinking 2507
Qwen · bartowski
Params4B
Context256K
Best fitQ8_0 6.0 GB
DE
DeepSeek V4.1 Flash 1017 ELVEZ Q2
DeepSeek · elvezjp
Params485M
Context—
Smallest1.0 GB
PH
phonon 40m fr
Phonon · gradium
Params51M
Context—
Smallest0.1 GB
LL
4 alokk llama 3 8b English to Hinglish
Llama · RichardErkhov
Params8B
Context8K
Best fitQ3_K_L 5.9 GB