GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
NE
NeuralLemon 7B
Neurallemon · backyardai
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
QW
FLUX 3 Image Klein TE Uncensored
Qwen · AtomicChat
Params4.02B
Context256K
Best fitQ8_0 6.0 GB
LL
tachyphylaxis Meta Llama 3.1 Chat Uncensored
Llama · RichardErkhov
Params15M
Context—
Best fitIQ3_S 1.6 GB
QW
Qwen3 VL 4B Instruct C_Abliterated v2
Qwen · mradermacher
Params4B
Context256K
Best fitQ8_0 6.0 GB
QW
Satyr V0.1 4B
Qwen · PantheonUnbound
Params4B
Context256K
Best fitQ8_0 6.0 GB
QU
Quyet 1.0 Medium
Quyet · mradermacher
Params4.33B
Context256K
Best fitQ6_K 5.2 GB
LL
KISTI KONI KONI Llama3 8B Instruct 20240729
Llama · RichardErkhov
Params8B
Context—
Best fitQ2_K 2.7 GB
NO
2B_or_not_2B
Not · mradermacher
Params2.5B
Context8K
Best fitQ6_K 2.7 GB
QW
Dolphin3.0 Qwen2.5 1.5B
Qwen · bartowski
Params1.54B
Context128K
Best fitF16 3.8 GB
QW
Qwen3-Vl-4B-Thinking
Qwen · unsloth
Params4B
Context256K
Best fitQ8_0 6.0 GB
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
QW
Qwen3.8 4B Distilled
Qwen · Ma7ee7
Params4B
Context256K
Best fitQ8_0 6.0 GB
OL
olmOCR 2 7B 1025
OLMo · bartowski
Params7.62B
Context125K
Best fitQ4_1 5.8 GB
QW
Qwen3 235B A22B Abliterated
Qwen · mradermacher
Params235B
Context40K
Q4_K_M142.2 GB
LA
Laguna S 2.1 Uncensored APEX
Laguna · SC117
Params118B
Context1024K
Best fitBF16 2.9 GB
JE
Jev-Style-0.8B-Decision-v3
Jev · chaoliangUNSW
Params752M
Context256K
Best fitF16 2.4 GB
QW
RuadaptQwen3 4B Instruct
Qwen · RefalMachine
Params4B
Context256K
Best fitQ8_0 6.0 GB
LE
lev
Lev · ggml-org
Params4.2B
Context256K
Best fitQ4_K_M 4.6 GB
QW
Qwen3.6 35B A3B abliterated v2
Qwen · LMLiutenant
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen3.5 2B Uncensored Abliterix SOM
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
CO
Yi Coder 1.5B Chat
Coder · bartowski
Params1.48B
Context128K
Best fitF16 5.1 GB
DE
DeepSeek R1 Distill Llama 8B Abliterated
DeepSeek · mradermacher
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
NO
North Mini Code 1.0
North · bartowski
Params30.5B
Context488K
Q4_K_M18.7 GB
MI
MiniCPM5 2B Abliterated Uncensored
MiniCPM · mondk
Params2.52B
Context128K
Best fitF16 5.9 GB
QW
Qwen3.8 2B Uncensored
Qwen · atakhadivi
Params1.88B
Context256K
Best fitQ4_K_M 2.2 GB
OC
Occamy 1.0 APEX I MiniPlus V2.1 Abliterated
Occamy · IsValorum
Params34.7B
Context256K
Best fitQ8_0 3.2 GB
GL
GLM 5.3 Flash UNCENSORED NVFP4 DFlash2
GLM · Solstice-AI
Params1.17B
Context1024K
Best fitBF16 3.0 GB
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · lmstudio-community
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
PH
Phi 3 mini 4k instruct update2024 07 03 IMat
Phi · legraphista
Params3.82B
Context4K
Best fitIQ4_NL 5.9 GB
WI
Wingless_Imp_8B
Wingless · mradermacher
Params8B
Context128K
Best fitQ3_K_L 5.9 GB