GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
GE
Gemma 4 E2B It Uncensored MAX
Gemma · mradermacher
Params4.65B
Context128K
Best fitQ6_K 4.9 GB
FA
Falcon-H1-0.5B-Instruct
Falcon · unsloth
Params521M
Context16K
Best fitBF16 1.7 GB
QW
Qwopus3.5 4B v3
Qwen · Jackrong
Params4.2B
Context256K
Best fitQ6_K 5.0 GB
QW
Qwen3 VL 32B Thinking
Qwen · bartowski
Params32.8B
Context256K
Q4_K_M19.8 GB
OP
openbuddy-zero-14b-v22.3-32k
Openbuddy · mradermacher
Params14B
Context32K
Best fitIQ2_XS 6.0 GB
SM
SmolVLM2 2.2B Instruct Agentic GUI
Smolvlm · bartowski
Params1.81B
Context8K
Best fitBF16 5.7 GB
MA
Marco Mini Instruct
Marco · mradermacher
Params17.3B
Context32K
Best fitIQ2_XXS 6.0 GB
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GR
granite embedding 125m english
Granite · bartowski
Params124M
Context512
Best fitF16 1.1 GB
ST
stable-code-instruct-3b
Stable · stabilityai
Params2.8B
Context16K
Best fitQ5_K_M 5.2 GB
LL
LongWriter Llama3.1 8b
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
QW
Qwen3.8 Flash Next MTP Q8 0
Qwen · agentionai
Params3.88B
Context256K
Best fitQ8_0 5.5 GB
MI
Clinical BR Mistral 7B v0.2
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
ME
models--automerger--MeliodasM7-7B
Meliodas · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
CA
CaptainErisNebula 12B Chimera v1.1
Captainerisnebula · Lewdiculous
Params12.2B
Context1000K
Q4_K_M7.5 GB
DA
pocketdoc_dans-totsirocco-7b
Dans · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
QW
Qwen3.8 Flash Next
Qwen · kingjones777
Params3.88B
Context256K
Best fitQ8_0 5.5 GB
CY
Blackfrost-AI/CYBER-FROST-3.8-BF16
Cyber · 0xSojalSec
Params177B
Context256K
Best fitQ8_0 5.5 GB
QW
dolphin 2.9.2 qwen2 7b
Qwen · QuantFactory
Params7.62B
Context128K
Best fitQ4_1 5.8 GB
QW
Qwen3 30B A3B Claude 4.5 Opus High Reasoning 2507 ABLITERATED UNCENSORED V2
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
OL
OLMo 1B 0724 Hf
OLMo · mradermacher
Params1.28B
Context4K
Best fitQ6_K 2.6 GB
MI
MistralLite 7B
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
LA
Laguna XS 2.1
Laguna · mradermacher
Params33.4B
Context256K
Q4_K_M20.3 GB
ER
ERNIE 21B A3B Claude 4.5 High OPUS Thinking
Ernie · mradermacher
Params21.8B
Context128K
Best fitIQ1_M 5.9 GB
GE
anon 1b
Gemma · scratchyone
Params1B
Context32K
Best fitBF16 2.7 GB
KA
KAT Coder V2.5 Dev
Kat · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
OP
weyaxi_openhermes-2.5-neural-chat-7b-v3-2-7b
Openhermes · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
NA
Nanbeige4.3 33B
Nanbeige · mradermacher
Params33.1B
Context512K
Q4_K_M20.1 GB
SH
OS Shepherd 9B
Shepherd · bartowski
Params8.95B
Context256K
Best fitIQ3_XXS 5.8 GB
QW
Qwen3.5 9B
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB