GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
nl2sh 1.5b Q4 K M
Qwen · ThorOdinson246
Params1.54B
Context32K
Best fitQ4_K_M 1.7 GB
QW
Qwen3.5 0.8B Artemis
Qwen · turintech
Params752M
Context256K
Best fitQ5_K_M 1.5 GB
LF
LFM2.5-350M
Lfm · LiquidAI
Params354M
Context125K
Best fitBF16 1.7 GB
FL
FLUX.2 klein 4B
Flux · leejet
Params3.88B
Context—
Best fitQ8_0 5.3 GB
QW
Qwen3.8 Flash Next
Qwen · bartowski
Params177B
Context256K
Best fitQ8_0 5.4 GB
MI
MiniCPM o 4 5
MiniCPM · openbmb
Params8.2B
Context40K
Best fitF16 2.9 GB
CP
CPH Community 7B
Cph · CypherTroopers
Params7B
Context—
Best fitQ4_K_M 0.5 GB
QW
Qwen3.8 35B A3B Distill
Qwen · IsValorum
Params34.7B
Context256K
Best fitQ8_0 3.2 GB
GE
Gemma 3 1b It
Gemma · lmstudio-community
Params1B
Context32K
Best fitQ8_0 1.8 GB
BO
Bonsai-1.7B
Bonsai · prism-ml
Params1.72B
Context32K
Best fitQ1_0 1.7 GB
PA
PaddleOCR VL 1.6
Paddleocr · PaddlePaddle
Params467M
Context128K
Smallest0.9 GB
QW
Qwen3 4B Instruct 2507
Qwen · lmstudio-community
Params4B
Context256K
Best fitQ8_0 6.0 GB
QW
Qwen3-Vl-2B-Instruct
Qwen · unsloth
Params1.72B
Context256K
Best fitBF16 4.9 GB
QW
Qwen2.5 Coder 3B Instruct AWQ
Qwen · Qwen
Params3.4B
Context32K
Best fitQ8_0 4.4 GB
QW
Qwen3.8 27B Humanlike Chat
Qwen · LessThanThreeAI
Params26.9B
Context256K
Q4_K_M16.6 GB
GE
Gemma-3-270M-It
Gemma · unsloth
Params268M
Context32K
Best fitF16 1.2 GB
GE
Gemma-4-E2B-Uncensored-HauhauCS-Aggressive
Gemma · HauhauCS
Params4.65B
Context128K
Best fitQ6_K 5.0 GB
QW
Qwen2.5 Omni 3B
Qwen · ggml-org
Params3.4B
Context32K
Best fitQ8_0 4.4 GB
QW
Qwen3.6-27B
Qwen · ggml-org
Params26.9B
Context256K
Best fitQ8_0 5.8 GB
QW
Huihui Qwen3 Coder 30B A3B Instruct Abliterated
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
HE
Hermes 4 14B
Hermes · bartowski
Params14.8B
Context40K
Q4_K_M9.0 GB
QW
Qwen3.5-9B
Qwen · unsloth
Params9.2B
Context256K
Best fitQ2_K_XL 6.0 GB
NE
Nemotron 3 Embed 1B Q4 K M
Nemotron · zenmagnets
Params1.14B
Context256K
Best fitQ4_K_M 1.8 GB
GE
Gemma 3 4b It
Gemma · ggml-org
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
NA
Nanbeige4.3 33B
Nanbeige · bartowski
Params33.1B
Context512K
Q4_K_M20.8 GB
GE
gemma 4 31B it qat
Gemma · unsloth
Params480M
Context128K
Best fitBF16 3.6 GB
GR
Granite 4.0 1B BE
Granite · taronaeo
Params1.63B
Context—
Best fitBF16 4.2 GB
GE
gemma-4-12B-it
Gemma · ggml-org
Params11.9B
Context256K
Best fitBF16 2.4 GB
QW
Qwen2.5 1.5B Instruct
Qwen · bartowski
Params1.54B
Context32K
Best fitF16 3.8 GB
QW
Qwen2.5 VL 7B Instruct
Qwen · ggml-org
Params7.62B
Context125K
Best fitQ4_K_M 5.7 GB