GGUF models that fit in 12 GB
171517 GGUF language models have at least one quant whose estimated memory use is 12 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
LF
LFM2.5 8B A1B Hermes Agentic Coder Abliterated v3
Lfm · mradermacher
Params8.47B
Context125K
Best fitQ6_K 9.1 GB
LU
Lumimaid Magnum v4 12B
Lumimaid · bartowski
Params12.2B
Context128K
Best fitQ6_K 11.9 GB
QW
DPO Trained Qwen2.5 Python Coder 7B
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
CH
Chaotic Order 24B V5
Chaotic · mradermacher
Params23.6B
Context128K
Best fitIQ3_XS 11.7 GB
QW
Qwen3.5 2B Uncensored Zero Refusal SOMA
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
QW
Qwen3 Coder Next Uncensored Heretic
Qwen · mradermacher
Params79.7B
Context256K
Q4_K_M48.5 GB
MI
Tamil-Mistral-7B-v0.1
Mistral · RichardErkhov
Params7.39B
Context32K
Best fitQ8_0 9.4 GB
DE
Deacon-34B
Deacon · mradermacher
Params34.4B
Context4K
Best fitIQ2_XXS 11.8 GB
YU
Ruach Studio Models v2
Yue · goldhub
Params632M
Context32K
Best fitQ8_0 9.1 GB
CO
Yi Coder 1.5B
Coder · lmstudio-community
Params1.48B
Context128K
Best fitQ8_0 3.7 GB
NE
Nemotron 3.5 Embed 8B BF16
Nemotron · W4D
Params7.95B
Context256K
Best fitQ8_0 10.1 GB
RE
REBEL-OpenChat-3.5
Rebel · mradermacher
Params7.24B
Context8K
Best fitQ6_K 7.5 GB
MI
Sakura MiniCPM V 4.7 1B Uncensored
MiniCPM · webmp3
Params1.01B
Context256K
Best fitQ8_0 2.0 GB
MU
Muse Glimmer 30B AZ Focus
Muse · mradermacher
Params27.9B
Context128K
Best fitQ2_K 11.6 GB
QW
Qwen2.5 Coder 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ8_0 9.1 GB
QW
Qwen3 14B NSFW FDA
Qwen · mradermacher
Params14.8B
Context32K
Best fitQ4_1 11.2 GB
QW
Qwen3 1.7B
Qwen · rippertnt
Params2.03B
Context40K
Best fitQ4_K_M 2.7 GB
LL
ultravox v0 5 llama 3 1 8b
Llama · ggml-org
Params8B
Context128K
Best fitQ4_K_M 6.5 GB
GP
SpaceQA Gpt Oss 20b
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
QW
Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-BF16
Qwen · Andycurrent
Params4.21B
Context256K
Best fitBF16 10.0 GB
LF
LFM2 24B A2B
Lfm · bartowski
Params23.8B
Context125K
Best fitQ2_K 11.5 GB
CY
Cydonia 22B v1.2
Cydonia · bartowski
Params22.2B
Context32K
Best fitQ3_K_S 12.0 GB
QW
Qwen3 0.6B
Qwen · NobodyWho
Params752M
Context32K
Best fitQ4_K_M 1.9 GB
PH
Unsloth Phi 3.5 Mini Instruct
Phi · legraphista
Params3.82B
Context128K
Best fitBF16 11.4 GB
IN
Internlm2 Math Plus 20b
InternLM · mradermacher
Params19.9B
Context8K
Best fitQ3_K_M 11.8 GB
QW
Suri Qwen 3.8 27B Neuro
Qwen · mradermacher
Params27B
Context256K
Best fitIQ2_S 12.0 GB
QW
Qwen3 VL 4B Instruct Uncensored
Qwen · mradermacher
Params4B
Context256K
Best fitF16 9.8 GB
QW
FlashModel Qwen 3.6 35B A3B Maxed
Qwen · flashback2k
Params35.5B
Context256K
Q4_K_M21.7 GB
OP
OpenELM-3B-Instruct
Openelm · SanctumAI
Params3.04B
Context2K
Best fitF16 7.3 GB
KN
KnowLine 9B Gen1
Knowline · mradermacher
Params9.2B
Context256K
Best fitQ8_0 11.4 GB