GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
VO
VoxCPM-0.5B
Voxcpm · bluryar
Params728M
Context32K
Best fitF16 5.4 GB
MI
uukuguy_speechless-mistral-dolphin-orca-platypus-samantha-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
HE
L3.1 RP Hero InBetween 8B
Hero · DavidAU
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
OM
OmniDimen 2 20B Emotion
Omnidimen · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
IC
IceEspressoRPv2 7b i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
LL
VersatiLlama Llama 3.2 3B Instruct Abliterated
Llama · QuantFactory
Params3.21B
Context128K
Best fitQ8_0 4.9 GB
MI
teknium_mistral-trismegistus-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
LL
Llama-3-Groq-8B-Tool-Use
Llama · bartowski
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
MI
janhq_mistral-7b-instruct-v0.2-dare
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
MI
snap1 2B
MiniCPM · logitlab
Params2.52B
Context128K
Best fitBF16 5.9 GB
LL
Llama 3.1 8B Ultra Instruct
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
JE
Jebadiah 4b v2
Jebadiah · frontier-infra
Params4.33B
Context256K
Best fitQ5_K_M 4.8 GB
PY
Pythia 1b
Pythia · mradermacher
Params1.01B
Context2K
Best fitF16 3.6 GB
QW
X Coder SFT Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ4_1 5.8 GB
QW
Qwen2.5 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
FA
Falcon3 10B Instruct Abliterated
Falcon · bartowski
Params10.3B
Context32K
Best fitQ2_K 5.8 GB
LL
Llama3.1 8B PlumCode
Llama · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
HE
L3.1 RP Hero BigTalker 8B
Hero · DavidAU
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
PH
Phi 3 mini 128k instruct
Phi · mmnga
Params3.82B
Context128K
Best fitIQ4_NL 5.9 GB
MI
Roleplay-Mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
NA
Nanbeige4.1 3B
Nanbeige · mradermacher
Params3.93B
Context256K
Best fitQ8_0 5.2 GB
QW
Qwen3.5 9B Claude 4.6 HighIQ THINKING HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
LI
LightOnOCR 2 1B
Lightonocr · noctrex
Params596M
Context16K
Best fitBF16 2.6 GB
PH
Phi 3.5 Mini ChatML
Phi · mradermacher
Params3.82B
Context128K
Best fitQ4_K_S 5.9 GB
LI
LightOnOCR 2 1B Ocr Soup
Lightonocr · noctrex
Params596M
Context16K
Best fitBF16 2.6 GB
PH
Phi 3.5 Mini instruct_Uncensored
Phi · mradermacher
Params3.82B
Context128K
Best fitQ4_0 5.9 GB
QW
Qwen3 4B Uncensored
Qwen · mradermacher
Params4B
Context40K
Best fitQ8_0 6.0 GB
BA
BananaMind Translate V3.5 Pro
Bananamind · mradermacher
Params7.46B
Context128K
Best fitIQ3_XXS 5.9 GB
OR
Orpheus 3b FT Q8 0
Orpheus · lex-au
Params3.78B
Context128K
Best fitQ8_0 5.5 GB
OP
weyaxi_openhermes-2.5-neural-chat-v3-3-slerp
Openhermes · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB