GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
VO
VoxCPM-0.5B
Voxcpm · bluryar
Params728M
Context32K
Best fitF16 5.4 GB
GGUF
MI
uukuguy_speechless-mistral-dolphin-orca-platypus-samantha-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Coding
HE
L3.1 RP Hero InBetween 8B
Hero · DavidAU
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
Roleplay
OM
OmniDimen 2 20B Emotion
Omnidimen · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
GGUF
IC
IceEspressoRPv2 7b i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
LL
VersatiLlama Llama 3.2 3B Instruct Abliterated
Llama · QuantFactory
Params3.21B
Context128K
Best fitQ8_0 4.9 GB
GGUF
MI
teknium_mistral-trismegistus-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
LL
Llama-3-Groq-8B-Tool-Use
Llama · bartowski
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
Tool use
MI
janhq_mistral-7b-instruct-v0.2-dare
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
MI
snap1 2B
MiniCPM · logitlab
Params2.52B
Context128K
Best fitBF16 5.9 GB
GGUF
LL
Llama 3.1 8B Ultra Instruct
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GGUF
JE
Jebadiah 4b v2
Jebadiah · frontier-infra
Params4.33B
Context256K
Best fitQ5_K_M 4.8 GB
GGUF
PY
Pythia 1b
Pythia · mradermacher
Params1.01B
Context2K
Best fitF16 3.6 GB
GGUF
QW
X Coder SFT Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ4_1 5.8 GB
Coding
QW
Qwen2.5 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
GGUF
FA
Falcon3 10B Instruct Abliterated
Falcon · bartowski
Params10.3B
Context32K
Best fitQ2_K 5.8 GB
GGUF
LL
Llama3.1 8B PlumCode
Llama · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
Coding
HE
L3.1 RP Hero BigTalker 8B
Hero · DavidAU
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
Roleplay
PH
Phi 3 mini 128k instruct
Phi · mmnga
Params3.82B
Context128K
Best fitIQ4_NL 5.9 GB
GGUF
MI
Roleplay-Mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Roleplay
NA
Nanbeige4.1 3B
Nanbeige · mradermacher
Params3.93B
Context256K
Best fitQ8_0 5.2 GB
GGUF
QW
Qwen3.5 9B Claude 4.6 HighIQ THINKING HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
ReasoningRoleplay
LI
LightOnOCR 2 1B
Lightonocr · noctrex
Params596M
Context16K
Best fitBF16 2.6 GB
Vision
PH
Phi 3.5 Mini ChatML
Phi · mradermacher
Params3.82B
Context128K
Best fitQ4_K_S 5.9 GB
GGUF
LI
LightOnOCR 2 1B Ocr Soup
Lightonocr · noctrex
Params596M
Context16K
Best fitBF16 2.6 GB
Vision
PH
Phi 3.5 Mini instruct_Uncensored
Phi · mradermacher
Params3.82B
Context128K
Best fitQ4_0 5.9 GB
GGUF
QW
Qwen3 4B Uncensored
Qwen · mradermacher
Params4B
Context40K
Best fitQ8_0 6.0 GB
GGUF
BA
BananaMind Translate V3.5 Pro
Bananamind · mradermacher
Params7.46B
Context128K
Best fitIQ3_XXS 5.9 GB
GGUF
OR
Orpheus 3b FT Q8 0
Orpheus · lex-au
Params3.78B
Context128K
Best fitQ8_0 5.5 GB
Tool use
OP
weyaxi_openhermes-2.5-neural-chat-v3-3-slerp
Openhermes · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Tool use

Other memory sizes

Keep browsing