GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
GE
Gemma 3 270m It Qat Q4_0 Unquantized
Gemma · ggml-org
Params268M
Context32K
Best fitQ4_0 0.9 GB
KE
Kev-0.8B
Kev · ggml-org
Params753M
Context256K
Best fitBF16 2.4 GB
GE
Gemma-3N-E2B-It
Gemma · unsloth
Params4.46B
Context32K
Best fitQ8_0 5.8 GB
QW
Qwen3.8 27B Heretic Ara
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
RA
RakutenAI 7B Chat
Rakutenai · mradermacher
Params7.37B
Context32K
Best fitQ4_K_M 6.0 GB
QW
Qwen2.5-VL-7B-Instruct
Qwen · Mungert
Params7.6B
Context125K
Best fitQ4_K_M 5.8 GB
LL
Llama-3-Groq-8B-Tool-Use
Llama · lmstudio-community
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
QW
Qwen3.8 27B Heretic Abliterated Uncensored
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
TI
TildeOpen 30B Instruct LV
Tildeopen · mradermacher
Params30.7B
Context64K
Q4_K_M18.5 GB
QW
Qwen3.5 4B MultiDistill 3xMerge UNCENSORED Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
FL
flux 3 image klein prompt upsampler
Flux · prithivMLmods
Params8.95B
Context256K
Best fitQ2_K 5.4 GB
FR
FrogNano 4B Heretic
Frognano · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
QW
Qwen3.5 2B Base Heretic
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
GR
Granite 3.1 1b A400M Instruct
Granite · bartowski
Params1.33B
Context128K
Best fitF16 3.6 GB
QW
Qwen3.8 Flash Next Rulith EXL3
Qwen · Rulith
Params177B
Context256K
Smallest2.6 GB
YI
01-ai_yi-6b-200k
Yi · TheBloke
Params6.06B
Context195K
Best fitQ6_K 6.0 GB
TE
Ternary Bonsai 2 27B DFlash2
Ternary · ProCreations
Params1.92B
Context256K
Best fitQ8_0 2.7 GB
OX
OxCoder 9B
Oxcoder · prithivMLmods
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
QW
Greko 1.7b Beta Merged
Qwen · geolazagr
Params1.72B
Context40K
Best fitIQ4_NL 2.5 GB
GR
granite-8b-code-instruct
Granite · mradermacher
Params8.05B
Context4K
Best fitQ3_K_L 6.0 GB
GE
Gemmasutra Small 4B v1
Gemma · TheDrummer
Params3.86B
Context8K
Best fitQ8_0 6.0 GB
GE
Gemma 4 E4B it Artemis
Gemma · turintech
Params7.46B
Context128K
Best fitQ4_0 4.8 GB
QW
Qwen2.5 VL 7B Instruct
Qwen · DhruvalLabs
Params7.62B
Context125K
Best fitQ4_K_M 5.7 GB
YI
Yi-1.5-9B-Chat
Yi · bartowski
Params8.83B
Context4K
Best fitQ3_K_L 6.0 GB
QW
Qwen3-8B-128K
Qwen · unsloth
Params8.19B
Context128K
Best fitQ3_K_XL 6.0 GB
OR
Ornith1.0 9B 3MPER0RR Abliterated
Ornith · mradermacher
Params8.95B
Context256K
Best fitIQ3_S 6.0 GB
QW
Qwen3 VL Reranker 2B
Qwen · mradermacher
Params1.72B
Context256K
Best fitQ6_K 2.9 GB
QW
Qwen3.8 9b Hermes Function Calling v2
Qwen · mradermacher
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
QW
Qwen3.8 Flash Next
Qwen · TonyMFJ
Params51.2B
Context256K
Best fitQ4_K_M 4.1 GB
MI
Mistral-Large-4-1T-A52B-DSpark
Mistral · unsloth
Params1050B
Context1024K
Best fitBF16 4.8 GB