GGUF models that fit in 12 GB
171517 GGUF language models have at least one quant whose estimated memory use is 12 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
ST
Starcoder2 3b
StarCoder · mradermacher
Params3.03B
Context16K
Best fitQ6_K 3.2 GB
DE
Deepseek Coder 7b Instruct v1.5
DeepSeek · mradermacher
Params6.91B
Context4K
Best fitQ6_K 10.2 GB
MA
MAI Gym 9B
Mai · mradermacher
Params8.95B
Context256K
Best fitQ6_K 9.0 GB
GR
Groot 19B A3B 261023
Groot · mradermacher
Params19B
Context256K
Best fitIQ4_XS 11.6 GB
QW
Ostrich 27B 261008 Qwen3.8 Abliterated
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
CA
CalibForge 35B A3B
Calibforge · mradermacher
Params34.7B
Context256K
Best fitIQ2_S 11.8 GB
QW
ThinCtx Qwen3.5 2B
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
QW
Qwen3.5 DeltaCoder 9B
Qwen · danielcherubini
Params8.95B
Context256K
Best fitQ8_0 11.1 GB
AC
ActIO-UI-7B-RLVR
Actio · mradermacher
Params7.62B
Context125K
Best fitQ6_K 7.2 GB
LL
Ornith 1.0 35B
Llama · LordNeel
Params34.7B
Context256K
Best fitBF16 4.9 GB
D1
d1 3B
AtomicChat
Params2.7B
Context32K
Best fitBF16 7.9 GB
LB
LBS 27B
Lbs · bartowski
Params26.9B
Context256K
Best fitIQ2_XS 11.5 GB
PE
Peach 9B 8k Roleplay
Peach · bartowski
Params8.83B
Context16K
Best fitQ8_0 10.7 GB
LL
chargoddard llama 2 26b trenchcoat stack
Llama · RichardErkhov
Params25.7B
Context—
Best fitQ4_K_S 12.0 GB
MO
L3 4X8B MOE Dark Planet Infinite 25B
Moe · DavidAU
Params24.9B
Context8K
Best fitQ2_K 10.9 GB
HO
K2 Horizon 7B UnrealEngine
Horizon · mradermacher
Params9B
Context512K
Best fitQ6_K 9.1 GB
GE
Gemma 4 26B A4B It Uncensored Heretic
Gemma · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
GE
Gemma 4 E4B It Uncensored Heretic
Gemma · OS-Software
Params7.46B
Context128K
Best fitQ8_K_XL 10.5 GB
QW
Qwen3.6 35B A3B Claude 4.7 Opus Reasoning Distilled
Qwen · mradermacher
Params34.7B
Context256K
Best fitIQ2_S 11.8 GB
MI
Mistral Nemo 12B ArliAI RPMax v1.1
Mistral · bartowski
Params12.2B
Context1000K
Best fitQ6_K 11.9 GB
QW
Qwen3 VL 8B Thinking Unredacted MAX
Qwen · prithivMLmods
Params8.19B
Context256K
Best fitQ8_0 10.4 GB
QW
Qwen3.8 4b Heretic Trial23
Qwen · hakergeniusz
Params4.21B
Context256K
Best fitBF16 10.0 GB
TO
tokyotech llm Swallow 7b NVE instruct hf
Tokyotech · RichardErkhov
Params6.74B
Context4K
Best fitQ6_K 10.3 GB
PR
Bu 30b A3B Preview
Preview · bartowski
Params30.5B
Context256K
Best fitIQ2_M 11.2 GB
EX
EXAONE 3.5 7.8B Instruct
Exaone · lmstudio-community
Params7.8B
Context32K
Best fitQ8_0 9.9 GB
VE
VeriLoop E2
Veriloop · tsinghua-sigs-robot-lab
Params27B
Context256K
Best fitBF16 8.7 GB
ME
Meltemi 7B Instruct v1.5
Meltemi · mradermacher
Params7.48B
Context32K
Best fitQ6_K 7.7 GB
EX
EXAONE 3.5 7.8B Instruct
Exaone · bartowski
Params7.8B
Context32K
Best fitQ8_0 9.9 GB
QW
Qwen3.8-Flash-CIRU-STRIX-IU4-MTP
Qwen · jcbtc
Params126B
Context256K
Best fitQ8_0 5.5 GB
GR
Granite 8b Code Instruct
Granite · duyntnet
Params8.05B
Context4K
Best fitQ8_0 10.3 GB