GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
RW
rwkv 6 world 1b6
Rwkv · latestissue
Params1.6B
Context1024K
Best fitBF16 4.1 GB
LL
Llama 3.1 SuperNova Lite
Llama · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
QW
Qwen3.5 9B DFlash
Qwen · magnitudedev
Params1.29B
Context40K
Best fitQ8_0 2.1 GB
MI
bitext Mistral 7B Customer Support
Mistral · RichardErkhov
Params7.24B
Context32K
Best fitQ4_K 5.9 GB
MA
Magidonia 24B v4.2.0
Magidonia · bartowski
Params23.6B
Context128K
Q4_K_M14.3 GB
IN
InternVL3_5 8B
Internvl · mradermacher
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
MI
Voxtral-Mini-4B-Realtime-2602
Mistral · ggml-org
Params3.43B
Context128K
Best fitQ8_0 5.0 GB
NO
Novel Humanize 4b
Novel · mradermacher
Params4.02B
Context40K
Best fitQ6_K 5.0 GB
NS
NSFW 3B
Nsfw · QuantFactory
Params2.8B
Context4K
Best fitQ6_K 5.5 GB
TA
UI TARS 7B DPO
Tars · lmstudio-community
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
TO
ToolSpace 8B
Toolspace · mradermacher
Params8.19B
Context64K
Best fitQ3_K_M 5.8 GB
TI
TinyLlama-1.1B-DavidAttenborough
Tinyllama · mradermacher
Params1.1B
Context4K
Best fitQ6_K 1.6 GB
KO
Konstanta V4 Alpha 7B
Konstanta · mradermacher
Params7.24B
Context8K
Best fitQ4_K_S 5.9 GB
GR
MN GRAND Gutenberg Lyra4 Lyra 12B DARKNESS
Grand · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
SE
Securecoder 30b Pro v3 Merged
Securecoder · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
GP
DMind-3
gpt-oss · Mungert
Params20.9B
Context128K
Q4_K_M16.0 GB
FR
Frontis MA1 35B
Frontis · bartowski
Params35.5B
Context256K
Q4_K_M21.9 GB
QW
Qwen3.6 35B A3B Brainwaves Nex Uncensored
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
QW
BigBubba Qwen3.6 27B
Qwen · bartowski
Params27.3B
Context256K
Q4_K_M17.8 GB
MI
Huihui MiniCPM V 4.6 Thinking Abliterated
MiniCPM · mradermacher
Params752M
Context256K
Best fitQ5_K_S 1.5 GB
QW
SignOfFour
Qwen · mradermacher
Params34.7B
Context32K
Q4_K_M21.2 GB
MI
rana shahroz mistral 7b roleplay alpaca merge
Mistral · RichardErkhov
Params7.24B
Context32K
Best fitQ4_K 5.9 GB
VU
VulnLLM R 7B
Vulnllm · mradermacher
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
QW
Qwen2 VL 2B Instruct
Qwen · second-state
Params1.54B
Context32K
Best fitF16 3.8 GB
QW
Qwen3.5 9B OmniCoder Claude Polaris
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
SO
SOLAR 10.7B v1.0
Solar · mradermacher
Params10.7B
Context4K
Best fitIQ2_M 5.8 GB
JA
Jan-Nano
Jan · unsloth
Params4B
Context40K
Best fitQ8_0 6.0 GB
GP
GPT OSS Cybersecurity 20B Merged
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
AN
Anime Girl 1.5B
Anime · mradermacher
Params1.54B
Context32K
Best fitQ6_K 2.0 GB
GE
gemma 4 E2B it qat q4 0 heretic
Gemma · FadedRedStar
Params4.63B
Context128K
Best fitQ4_0 3.2 GB