GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
GE
gemma-4-26B-A4B-it
Gemma · ggml-org
Params450M
Context256K
Best fitBF16 2.4 GB
QW
qwen2.5-0.5b-instruct
Qwen · Qwen
Params630M
Context32K
Q4_K_M0.5 GB
SP
Spark X2.5 4B
Spark · XHToken
Params4.1B
Context1024K
Best fitQ4_K_M 4.3 GB
SP
Spark X2.5 4B
Spark · abenzerps
Params4.1B
Context1024K
Best fitQ6_K 5.1 GB
TH
Thomson 1.0 Small
Thomson · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
OR
Ornith 1.5 9B OBLITERATED
Ornith · OBLITERATUS
Params9.2B
Context256K
Best fitQ2_K 5.5 GB
GE
Gemmable 4 12B MTP
Gemma · Mia-AiLab
Params11.9B
Context256K
Q4_K_M0.3 GB
LF
LFM2.5 Audio 1.5B JP
Lfm · LiquidAI
Params190M
Context125K
Best fitF32 5.3 GB
TE
Ternary Bonsai 8B
Ternary · prism-ml
Params8.19B
Context64K
Best fitQ2_0 4.0 GB
QW
Qwen2.5 Coder 7B Instruct GGUF
Qwen · Qwen
Params15B
Context128K
Best fitQ3_K_M 4.8 GB
JA
Jan-v3.5-4B
Jan · janhq
Params4.4B
Context256K
Best fitQ6_K 5.3 GB
HY
Hy MT2 1.8B
tencent
Params1.79B
Context256K
Best fitQ8_0 2.9 GB
LF
LFM2.5 1.2B Instruct
Lfm · LiquidAI
Params1.17B
Context125K
Best fitBF16 3.9 GB
QW
Qwen3.8 27B Uncensored Cyber agentic imatrix
Qwen · cyjin-yl
Params31B
Context—
Smallest0.0 GB
SP
Spark X2.5 4B Heretic jp
Spark · soyaakinohara
Params4.1B
Context1024K
Best fitQ6_K 5.1 GB
LF
LFM2.5 230M
Lfm · LiquidAI
Params230M
Context125K
Best fitBF16 1.4 GB
ME
Mellum2.1-12B-A2.5B-Thinking
Mellum · JetBrains
Params12.1B
Context128K
Best fitBF16 2.7 GB
QW
Bonsai-27B-dspark
Qwen · prism-ml
Params3.6B
Context4K
Best fitQ1_0 4.4 GB
KA
KAT Coder V2.5 Dev
Kat · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
MI
MiniCPM5 2B
MiniCPM · openbmb
Params2.52B
Context128K
Best fitF16 5.9 GB
QW
Qwen3.8 27B
Qwen · bartowski
Params27.3B
Context256K
Q4_K_M17.4 GB
QW
Huihui Qwen3.8 Flash Next abliterated v2
Qwen · huihui-ai
Params177B
Context—
Smallest1.2 GB
MI
MiniCPM V 4.7 35B A3B
MiniCPM · openbmb
Params34.7B
Context256K
Best fitF16 2.0 GB
GR
Granite 4.2 3b
Granite · ibm-granite
Params3.66B
Context128K
Best fitQ8_0 5.1 GB
GR
Granite 4.2 8b
Granite · ibm-granite
Params8.79B
Context128K
Best fitQ3_K_S 5.8 GB
LF
LFM2.5 Audio 1.5B
Lfm · LiquidAI
Params190M
Context125K
Best fitF16 3.0 GB
QW
Qwen3.5-0.8B
Qwen · unsloth
Params752M
Context256K
Best fitBF16 2.4 GB
QW
Ternary-Bonsai-2-2B
Qwen · prism-ml
Params1.88B
Context256K
Smallest0.4 GB
GP
gpt-oss-20b
gpt-oss · ggml-org
Params870M
Context128K
Best fitBF16 2.2 GB
GE
Gemma 3 1b It
Gemma · ggml-org
Params1B
Context32K
Best fitF16 2.7 GB