GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
TR
TranslatePsy AfriSLM 4B
Translatepsy · mradermacher
Params4.84B
Context256K
Best fitQ6_K 5.6 GB
MI
lemonilia_ashhlimarp-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
ON
OneJev 0.8B
Onejev · bartowski
Params1.01B
Context256K
Best fitBF16 2.9 GB
HO
K2 Horizon MoVA 36B A4B
Horizon · ngquocvinh
Params37.4B
Context512K
Q4_K_M22.4 GB
LL
Llama Leonida
Llama · mradermacher
Params8.2B
Context128K
Best fitQ3_K_L 6.0 GB
ME
models--automerger--T3qm7Mergerix-7B
Mergerix · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
ST
StartLux-Decision-v1.1-0.8B-Instant
Startlux · startlux-models
Params752M
Context256K
Best fitQ8_0 1.7 GB
AN
Andromeda 8B Coder i1
Andromeda · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
YA
models--automerger--Experiment28Yam-7B
Yam · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
ST
stablelm zephyr 3b
Stablelm · TheBloke
Params2.8B
Context4K
Best fitQ6_K 5.5 GB
EV
EAN 7B i1
Evan · mradermacher
Params7.62B
Context125K
Best fitQ4_1 5.8 GB
SU
SummLlama3 8B i1
Summllama · mradermacher
Params8.03B
Context8K
Best fitIQ4_XS 6.0 GB
QW
Trohrbaugh Qwen3.5 122B A10B Heretic
Qwen · mradermacher
Params122B
Context256K
Q4_K_M74.2 GB
QW
Qwen3-4B-128K
Qwen · unsloth
Params4B
Context128K
Best fitQ8_0 6.0 GB
TI
Tiamat-7b-1.1-DPO
Tiamat · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
LL
Llama-3-Patronus-Lynx-8B-Instruct
Llama · mradermacher
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
RO
ROMA 7B
Roma · mradermacher
Params7.62B
Context32K
Best fitQ4_1 5.8 GB
GR
GRM 2.6 Plus 0628
Grm · bartowski
Params27.3B
Context256K
Q4_K_M17.8 GB
SM
SmolVLM Instruct
Smolvlm · ggml-org
Params1.81B
Context16K
Best fitF16 5.7 GB
KA
KAI 7B v0.1
Kai · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
LF
LFM2 1.2B Tool
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
MO
Mox Small 1
Mox · mradermacher
Params32.2B
Context64K
Q4_K_M19.5 GB
MI
MiniMax M2.1
Minimax · mradermacher
Params230B
Context192K
Q4_K_M138.3 GB
HU
Huihui Agents A1 Abliterated
Huihui · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
OX
OxCoder 9B
Oxcoder · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
OP
OpenCoder 1.5B Instruct
Opencoder · lmstudio-community
Params1.91B
Context4K
Best fitQ8_0 4.3 GB
HO
Holo 3.1 9B
Holo · prithivMLmods
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
MI
Devstral 2 123B Instruct 2512
Mistral · bartowski
Params125B
Context256K
Q4_K_M74.9 GB
QW
Remnant Qwen3 8b
Qwen · bartowski
Params8.19B
Context32K
Best fitQ3_K_M 5.8 GB
HE
L3.1 RP Hero Dirty Harry 8B
Hero · DavidAU
Params8B
Context128K
Best fitQ3_K_L 5.9 GB