GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Qwen2.5 1.5B Instruct
Qwen · QuantFactory
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
QW
Qwen3.6 35B A3B DFlash2
Qwen · aminya
Params526M
Context256K
Best fitBF16 1.8 GB
DI
DialFactSum ACU 8B
Dialfactsum · mradermacher
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
KA
Kadoe 4B
Kadoe · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
MI
UCLA AGI Mistral7B PairRM SPPO Iter3
Mistral · RichardErkhov
Params7.24B
Context32K
Best fitQ4_K 5.9 GB
QW
Qwen3 4B
Qwen · NobodyWho
Params4B
Context32K
Best fitQ4_K_M 4.2 GB
WI
Winterreise-m7
Winterreise · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
IN
models--automerger--Inex12Multiverseex26-7B
Inex · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
OC
Occamy 1.0
Occamy · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
GL
Poe 8B GLM5 Opus4.6 Sonnet4.5 Kimi Grok Gemini 3 Pro Preview HERETIC
GLM · mradermacher
Params8.19B
Context256K
Best fitQ3_K_M 5.8 GB
LL
Llama 3.2 3B MoE 4Expert
Llama · mradermacher
Params9.55B
Context128K
Best fitIQ3_M 5.8 GB
MI
Ministral 3b instruct
Mistral · mradermacher
Params3.32B
Context128K
Best fitQ8_0 4.5 GB
QW
Swift 1.5 Qwen3.8 27B Uncensored MTP
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
PY
Pythia 1.4b
Pythia · mradermacher
Params1.41B
Context2K
Best fitQ6_K 3.3 GB
SP
SpatialAxiom 9B
Spatialaxiom · bartowski
Params8.95B
Context256K
Best fitIQ3_XXS 5.8 GB
DE
DeepSeek R1 Distill Qwen 1.5B Heretic
DeepSeek · saidutta69
Params1.78B
Context128K
Best fitF16 4.3 GB
NA
NaruMOE-3x7B-v2
Narumoe · mradermacher
Params18.5B
Context32K
Best fitIQ1_M 5.8 GB
DE
deepseek coder 1.3B kexer
DeepSeek · QuantFactory
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
CO
beowolx_codeninja-1.0-openchat-7b
Codeninja · TheBloke
Params7.24B
Context8K
Best fitQ4_K_M 5.9 GB
QW
MiRL Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
PH
Athene Phi 3.5 mini instruct orpo
Phi · mradermacher
Params3.82B
Context128K
Best fitQ4_K_M 6.0 GB
TI
TinyLlama 1.1B Chat v1.0 imatrix
Tinyllama · duyntnet
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
LL
Aion RP Llama 3.1 8B
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
QW
Qwen3.5 2B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
ST
stabilityai japanese stablelm base gamma 7b
Stabilityai · RichardErkhov
Params7.24B
Context4K
Best fitQ4_K 5.9 GB
GE
Gemma 3 1b It Qat Q4_0 Unquantized
Gemma · ggml-org
Params1B
Context32K
Best fitQ4_0 1.4 GB
EL
Elephant 1
Elephant · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.9 GB
LU
dreamgen lucid v1 nemo
Lucid · bartowski
Params12.2B
Context128K
Best fitIQ2_S 6.0 GB
OV
OvisOCR2
Ovisocr · MDfox
Params752M
Context256K
Best fitBF16 2.4 GB
LI
LightOnOCR 3 0.8B
Lightonocr · mradermacher
Params752M
Context256K
Best fitF16 2.4 GB