GGUF models that fit in 12 GB
171517 GGUF language models have at least one quant whose estimated memory use is 12 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Qwen2.5 Coder 14B Instruct Uncensored Q4 K M
Qwen · roleplaiapp
Params14.8B
Context32K
Best fitQ4_K_M 11.1 GB
EK
Ekbasis 27B
Ekbasis · mradermacher
Params27B
Context256K
Best fitIQ2_S 12.0 GB
GE
gemmuguinho 636m
Gemmuguinho · Papuguinho
Params636M
Context32K
Best fitF16 1.9 GB
UN
ajibawa-2023_uncensored-jordan-7b
Uncensored · TheBloke
Params6.74B
Context2K
Best fitQ8_0 12.0 GB
KA
KansenSakura Eclipse RP 12b
Kansensakura · mradermacher
Params12.2B
Context1000K
Best fitQ6_K 11.9 GB
OP
openbuddy-yi1.5-34b-v21.3-32k
Openbuddy · mradermacher
Params34.4B
Context128K
Best fitIQ2_XXS 11.8 GB
LL
Llama 3.2 3B Instruct
Llama · mradermacher
Params3.61B
Context128K
Best fitQ6_K 4.4 GB
GL
Huihui GLM 4.6V Flash abliterated
GLM · AliBilge
Params9.4B
Context128K
Best fitQ8_0 10.8 GB
GE
gemma 4 E2B it UD japanese imatrix
Gemma · dahara1
Params4.65B
Context128K
Best fitQ8_K_XL 6.4 GB
MI
MiniCPM V 4.7 1B Heretic
MiniCPM · mradermacher
Params1.01B
Context256K
Best fitQ6_K 1.8 GB
QW
Qwen3-Vl-8B-Thinking-1M
Qwen · unsloth
Params8.19B
Context977K
Best fitQ8_0 10.4 GB
GR
Granite 8b Code Instruct 4k
Granite · RichardErkhov
Params8.05B
Context4K
Best fitQ8_0 10.3 GB
HY
HyperNova 60B
Hypernova · mradermacher
Params58.7B
Context128K
Q4_K_M40.7 GB
QW
WhereOPD Qwen3.5 4B
Qwen · mradermacher
Params4.84B
Context256K
Best fitQ6_K 5.6 GB
LL
Llama-3-Instruct-Neurona-8b-v2
Llama · mradermacher
Params8B
Context8K
Best fitQ6_K 8.2 GB
LF
LFM2.5 24B A2B Abliterated
Lfm · mradermacher
Params24B
Context128K
Best fitQ2_K 12.0 GB
ST
L3-8B-Stheno-v3.2
Stheno · DavidAU
Params8B
Context8K
Best fitQ8_0 10.1 GB
DE
DeepNarra-Paham-Experimental
Deepnarra · DeepNarra
Params952M
Context2K
Smallest0.3 GB
DE
Qwen3 30B A3B Deepseek Distill Instruct 2507
DeepSeek · mradermacher
Params30.5B
Context256K
Best fitQ2_K_S 11.8 GB
LL
nousresearch_yarn-llama-2-7b-64k
Llama · TheBloke
Params6.74B
Context64K
Best fitQ8_0 12.0 GB
MI
MiniCPM5-2B-Claude-Fable5-1-Thinking
MiniCPM · GnLOLot
Params2.52B
Context128K
Best fitF16 5.9 GB
QW
Qwen3 4B
Qwen · enacimie
Params4.02B
Context40K
Best fitQ4_K_M 6.7 GB
HO
Sakura K2 Horizon 7B
Horizon · webmp3
Params9B
Context512K
Smallest3.8 GB
MI
MixTaoTruthful-13B-slerp
Mixtaotruthful · mradermacher
Params12.9B
Context32K
Best fitQ5_K_M 10.7 GB
ST
Starcoder2 15b
StarCoder · mradermacher
Params16B
Context16K
Best fitQ4_K_M 11.0 GB
DE
Qwen2.5 MOE 2X1.5B DeepSeek Uncensored Censored 4B
DeepSeek · DavidAU
Params4.1B
Context128K
Best fitF16 8.9 GB
LL
Meta Llama 3 8B Instruct GGUF Imatrix smashed
Llama · PrunaAI
Params8B
Context8K
Best fitQ8_0 10.1 GB
QW
dnotitia_Smoothie Qwen3 4B
Qwen · Alcoft
Params4B
Context40K
Q4_K_M2.4 GB
SA
Saiga_Yandexgpt_8B_Sft_M4_D19_Hf
Saiga · IlyaGusev
Params8B
Context32K
Best fitQ8_0 10.1 GB
BO
Borealis Open 12b
Borealis · mradermacher
Params11.8B
Context128K
Best fitQ5_K_S 12.0 GB