GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 1.5B Instruct
Qwen · QuantFactory
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
GGUF
QW
Qwen3.6 35B A3B DFlash2
Qwen · aminya
Params526M
Context256K
Best fitBF16 1.8 GB
GGUF
DI
DialFactSum ACU 8B
Dialfactsum · mradermacher
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
ReasoningTool use
KA
Kadoe 4B
Kadoe · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
ReasoningTool useVision
MI
UCLA AGI Mistral7B PairRM SPPO Iter3
Mistral · RichardErkhov
Params7.24B
Context32K
Best fitQ4_K 5.9 GB
GGUF
QW
Qwen3 4B
Qwen · NobodyWho
Params4B
Context32K
Best fitQ4_K_M 4.2 GB
GGUF
WI
Winterreise-m7
Winterreise · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
IN
models--automerger--Inex12Multiverseex26-7B
Inex · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
OC
Occamy 1.0
Occamy · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
Tool useVision
GL
Poe 8B GLM5 Opus4.6 Sonnet4.5 Kimi Grok Gemini 3 Pro Preview HERETIC
GLM · mradermacher
Params8.19B
Context256K
Best fitQ3_K_M 5.8 GB
Vision
LL
Llama 3.2 3B MoE 4Expert
Llama · mradermacher
Params9.55B
Context128K
Best fitIQ3_M 5.8 GB
Coding
MI
Ministral 3b instruct
Mistral · mradermacher
Params3.32B
Context128K
Best fitQ8_0 4.5 GB
GGUF
QW
Swift 1.5 Qwen3.8 27B Uncensored MTP
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
GGUF
PY
Pythia 1.4b
Pythia · mradermacher
Params1.41B
Context2K
Best fitQ6_K 3.3 GB
GGUF
SP
SpatialAxiom 9B
Spatialaxiom · bartowski
Params8.95B
Context256K
Best fitIQ3_XXS 5.8 GB
Vision
DE
DeepSeek R1 Distill Qwen 1.5B Heretic
DeepSeek · saidutta69
Params1.78B
Context128K
Best fitF16 4.3 GB
ReasoningTool use
NA
NaruMOE-3x7B-v2
Narumoe · mradermacher
Params18.5B
Context32K
Best fitIQ1_M 5.8 GB
Roleplay
DE
deepseek coder 1.3B kexer
DeepSeek · QuantFactory
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
Coding
CO
beowolx_codeninja-1.0-openchat-7b
Codeninja · TheBloke
Params7.24B
Context8K
Best fitQ4_K_M 5.9 GB
Coding
QW
MiRL Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
ReasoningTool use
PH
Athene Phi 3.5 mini instruct orpo
Phi · mradermacher
Params3.82B
Context128K
Best fitQ4_K_M 6.0 GB
GGUF
TI
TinyLlama 1.1B Chat v1.0 imatrix
Tinyllama · duyntnet
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
GGUF
LL
Aion RP Llama 3.1 8B
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
Roleplay
QW
Qwen3.5 2B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
ReasoningRoleplay
ST
stabilityai japanese stablelm base gamma 7b
Stabilityai · RichardErkhov
Params7.24B
Context4K
Best fitQ4_K 5.9 GB
GGUF
GE
Gemma 3 1b It Qat Q4_0 Unquantized
Gemma · ggml-org
Params1B
Context32K
Best fitQ4_0 1.4 GB
GGUF
EL
Elephant 1
Elephant · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.9 GB
ReasoningTool useVision
LU
dreamgen lucid v1 nemo
Lucid · bartowski
Params12.2B
Context128K
Best fitIQ2_S 6.0 GB
GGUF
OV
OvisOCR2
Ovisocr · MDfox
Params752M
Context256K
Best fitBF16 2.4 GB
Vision
LI
LightOnOCR 3 0.8B
Lightonocr · mradermacher
Params752M
Context256K
Best fitF16 2.4 GB
Vision

Other memory sizes

Keep browsing