GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
SE
SexyGPT v3 Thinking Female
Sexygpt · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
ReasoningRoleplay
OU
OuteTTS 0.2 500M
Outetts · OuteAI
Params500M
Context32K
Q4_K_M0.4 GB
GGUF
QW
Qwen3 VL 8B Instruct Heretic
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ3_K_M 5.8 GB
Vision
SE
SenseNova U1 8B MoT SFT
Sensenova · mradermacher
Params9.35B
Context256K
Best fitIQ3_XS 6.0 GB
CodingReasoningTool use
GR
Kiesel v2
Granite · mradermacher
Params8.8B
Context128K
Best fitIQ3_M 5.9 GB
ReasoningTool use
LL
Akikawa 3.3 1b Roleplay 128K
Llama · Green-Eye
Params1.24B
Context128K
Best fitF16 3.2 GB
Roleplay
HO
K2 Horizon 3.7B
Horizon · NANI-Nithin
Params5.06B
Context512K
Best fitQ6_K 5.9 GB
GGUF
QW
Qwen3.5 4B
Qwen · gguf
Params4.33B
Context256K
Best fitQ6_K 5.2 GB
Vision
GR
Granite 4.0 H 1b
Granite · ibm-granite
Params1.46B
Context1024K
Best fitBF16 5.4 GB
GGUF
TE
NaviDC OCR
Teleocr · nandraj
Params752M
Context125K
Best fitF16 2.9 GB
Vision
GE
Huihui Gemma 3n E4B It Abliterated
Gemma · bartowski
Params6.9B
Context32K
Best fitQ5_K_S 6.0 GB
GGUF
QW
Qwen3.5 9B Heretic
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
GGUF
MO
MolOptAgent 7B
Moloptagent · mradermacher
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
GGUF
XL
xLAM 2 3b Fc R
Xlam · Salesforce
Params3.09B
Context32K
Best fitQ8_0 4.1 GB
Tool use
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · unsloth
Params1.54B
Context128K
Best fitF16 3.8 GB
Coding
WI
WizardCoder 15B V1.0 i1
Wizardcoder · mradermacher
Params15.8B
Context8K
Best fitIQ2_S 5.9 GB
Coding
SE
SEX_ROLEPLAY.v4 3.2 1B
Sex · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
RoleplayTool use
GE
Gemma 4 E2B
Gemma · mradermacher
Params4.65B
Context128K
Best fitQ6_K 4.9 GB
GGUF
MI
Mimo-Vl-7B-Rl
Mimo · unsloth
Params7.62B
Context125K
Best fitIQ4_XS 6.0 GB
Vision
FI
Fimbulvetr-11B-v2.1-16K
Fimbulvetr · mradermacher
Params10.7B
Context16K
Best fitIQ2_M 5.8 GB
GGUF
FL
flint
Flint · terelabs
Params39.8B
Context256K
Q4_K_M25.0 GB
ReasoningTool use
QW
Qwen3.8 27B Humanlike Chat 2.0
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
RoleplayTool use
QW
Qwen3.8 27B Uncensored
Qwen · jaromer
Params27.3B
Context256K
Best fitQ8_0 5.8 GB
GGUF
QW
Qwen3.8 9B Distill SLERP F451 Pro Writer Uncensored
Qwen · mradermacher
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
RoleplayVision
CL
Clef-Flash
Clef · turintech
Params9.08B
Context256K
Best fitQ3_K_S 5.7 GB
ReasoningTool use
HU
Huihui Ornith 1.5 9B Abliterated
Huihui · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
GGUF
PY
Pythia 2.8b Deduped
Pythia · mradermacher
Params2.78B
Context2K
Best fitQ6_K 5.5 GB
GGUF
SM
SmolLM2 1.7B Instruct
SmolLM · lmstudio-community
Params1.71B
Context8K
Best fitQ8_0 3.9 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · DavidAU
Params3.21B
Context128K
Best fitQ6_K 4.1 GB
Roleplay
QW
MiroThinker-v1.5-30B
Qwen · Mungert
Params30.5B
Context256K
Q4_K_M18.9 GB
Reasoning

Other memory sizes

Keep browsing