GGUF models that fit in 12 GB
171517 GGUF language models have at least one quant whose estimated memory use is 12 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
qwen2-0_5b-instruct
Qwen · Qwen
Params494M
Context32K
Q4_K_M0.4 GB
QW
Qwen2.5 Coder 3B Instruct Abliterated
Qwen · bartowski
Params3.09B
Context32K
Best fitF16 7.0 GB
GP
OpenAI Gpt Oss 20B Claude 4.5 Opus Heretic Uncensored
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
QW
Qwen2.5 Coder 7B Instruct Uncensored
Qwen · BlossomsAI
Params7.62B
Context32K
Best fitQ8_0 9.1 GB
GE
Gemma The Writer N Restless Quill 10B
Gemma · DavidAU
Params10B
Context8K
Best fitQ6_K 12.0 GB
LF
LFM2.5-1.2B-Thinking
Lfm · LiquidAI
Params1.17B
Context125K
Best fitBF16 3.9 GB
QW
Ingot XIII 27B Q2
Qwen · dendriteholdings
Params26.9B
Context256K
Smallest5.9 GB
MU
Music Flamingo Hf
Music · mradermacher
Params7.61B
Context32K
Best fitQ8_0 9.1 GB
MI
MiniCPM V 4.7 35B A3B
MiniCPM · mradermacher
Params34.7B
Context256K
Best fitIQ2_S 11.8 GB
QW
Qwen2.5 0.5B Instruct
Qwen · lmstudio-community
Params494M
Context32K
Best fitQ6_K 1.1 GB
QW
Qwen3-30B-A3B-Thinking-2507
Qwen · unsloth
Params30.5B
Context256K
Best fitIQ2_XXS 11.6 GB
QW
Qwen3.5-0.8B
Qwen · unsloth
Params773M
Context256K
Best fitBF16 2.5 GB
QW
Qwen3.8-27B
Qwen · 6block
Params27B
Context256K
Best fitQ8_0 5.8 GB
QW
Qwen3.5 4B NSFW ARA Heretic Literotica i1
Qwen · Sinbad-The-Sailor
Params4.2B
Context256K
Best fitQ6_K 5.0 GB
LL
Nidum Llama 3.2 3B Uncensored
Llama · lemuralabs
Params3.21B
Context128K
Best fitF16 7.9 GB
QW
Huihui Qwen3.5 9B Abliterated
Qwen · Abiray
Params8.95B
Context256K
Best fitQ8_0 11.1 GB
QW
Qwen3.6 35B A3B Anko
Qwen · bartowski
Params34.7B
Context256K
Best fitIQ2_XS 12.0 GB
NA
Nanonets-Ocr-S
Nanonets · unsloth
Params3.1B
Context125K
Best fitBF16 7.0 GB
DE
DeepSeek-Coder-V2-Lite-Instruct
DeepSeek · lmstudio-community
Params15.7B
Context160K
Best fitIQ4_XS 11.3 GB
QW
Qwen3.8 Flash Next Uncensored ik llama
Qwen · ji-farthing
Params177B
Context—
Smallest2.2 GB
QW
Qwen 3 8b Heretic
Qwen · DreamFast
Params8.19B
Context40K
Best fitQ8_0 10.4 GB
GA
GalTransl v4 4B 2601
Galtransl · SakuraLLM
Params4B
Context32K
Smallest2.8 GB
VE
UI Venus 2 9B
Venus · bartowski
Params8.95B
Context256K
Best fitQ8_0 11.1 GB
MU
Murasaki 4B v0.3
Murasaki · mradermacher
Params4.41B
Context32K
Best fitF16 10.5 GB
GE
Gemma 3 12b It
Gemma · ggml-org
Params11.8B
Context128K
Best fitQ4_K_M 11.0 GB
BO
Boulesis v2.1 26B A4B
Boulesis · mradermacher
Params26B
Context256K
Q4_K_M17.2 GB
QW
sakura-14b-qwen3-v1.5
Qwen · SakuraLLM
Params14.8B
Context32K
Smallest8.2 GB
QW
Qwopus3.8 27B Flash V2 Abliterated
Qwopus · mradermacher
Params27B
Context256K
Best fitIQ2_S 12.0 GB
FL
flux2 klein 4b uncensored text encoder
Flux · ponpoke
Params4B
Context—
Best fitF16 9.5 GB
LY
Lythri 7B A4B
Lythri · mradermacher
Params7.46B
Context128K
Best fitQ6_K 8.1 GB