GGUF models that fit in 12 GB

171517 GGUF language models have at least one quant whose estimated memory use is 12 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 Coder 14B Instruct Uncensored Q4 K M
Qwen · roleplaiapp
Params14.8B
Context32K
Best fitQ4_K_M 11.1 GB
Coding
EK
Ekbasis 27B
Ekbasis · mradermacher
Params27B
Context256K
Best fitIQ2_S 12.0 GB
GGUF
GE
gemmuguinho 636m
Gemmuguinho · Papuguinho
Params636M
Context32K
Best fitF16 1.9 GB
GGUF
UN
ajibawa-2023_uncensored-jordan-7b
Uncensored · TheBloke
Params6.74B
Context2K
Best fitQ8_0 12.0 GB
GGUF
KA
KansenSakura Eclipse RP 12b
Kansensakura · mradermacher
Params12.2B
Context1000K
Best fitQ6_K 11.9 GB
Roleplay
OP
openbuddy-yi1.5-34b-v21.3-32k
Openbuddy · mradermacher
Params34.4B
Context128K
Best fitIQ2_XXS 11.8 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · mradermacher
Params3.61B
Context128K
Best fitQ6_K 4.4 GB
GGUF
GL
Huihui GLM 4.6V Flash abliterated
GLM · AliBilge
Params9.4B
Context128K
Best fitQ8_0 10.8 GB
Vision
GE
gemma 4 E2B it UD japanese imatrix
Gemma · dahara1
Params4.65B
Context128K
Best fitQ8_K_XL 6.4 GB
Vision
MI
MiniCPM V 4.7 1B Heretic
MiniCPM · mradermacher
Params1.01B
Context256K
Best fitQ6_K 1.8 GB
RoleplayVision
QW
Qwen3-Vl-8B-Thinking-1M
Qwen · unsloth
Params8.19B
Context977K
Best fitQ8_0 10.4 GB
ReasoningVision
GR
Granite 8b Code Instruct 4k
Granite · RichardErkhov
Params8.05B
Context4K
Best fitQ8_0 10.3 GB
Coding
HY
HyperNova 60B
Hypernova · mradermacher
Params58.7B
Context128K
Q4_K_M40.7 GB
Tool use
QW
WhereOPD Qwen3.5 4B
Qwen · mradermacher
Params4.84B
Context256K
Best fitQ6_K 5.6 GB
GGUF
LL
Llama-3-Instruct-Neurona-8b-v2
Llama · mradermacher
Params8B
Context8K
Best fitQ6_K 8.2 GB
CodingTool use
LF
LFM2.5 24B A2B Abliterated
Lfm · mradermacher
Params24B
Context128K
Best fitQ2_K 12.0 GB
ReasoningTool use
ST
L3-8B-Stheno-v3.2
Stheno · DavidAU
Params8B
Context8K
Best fitQ8_0 10.1 GB
Roleplay
DE
DeepNarra-Paham-Experimental
Deepnarra · DeepNarra
Params952M
Context2K
Smallest0.3 GB
GGUF
DE
Qwen3 30B A3B Deepseek Distill Instruct 2507
DeepSeek · mradermacher
Params30.5B
Context256K
Best fitQ2_K_S 11.8 GB
GGUF
LL
nousresearch_yarn-llama-2-7b-64k
Llama · TheBloke
Params6.74B
Context64K
Best fitQ8_0 12.0 GB
GGUF
MI
MiniCPM5-2B-Claude-Fable5-1-Thinking
MiniCPM · GnLOLot
Params2.52B
Context128K
Best fitF16 5.9 GB
ReasoningTool use
QW
Qwen3 4B
Qwen · enacimie
Params4.02B
Context40K
Best fitQ4_K_M 6.7 GB
GGUF
HO
Sakura K2 Horizon 7B
Horizon · webmp3
Params9B
Context512K
Smallest3.8 GB
GGUF
MI
MixTaoTruthful-13B-slerp
Mixtaotruthful · mradermacher
Params12.9B
Context32K
Best fitQ5_K_M 10.7 GB
GGUF
ST
Starcoder2 15b
StarCoder · mradermacher
Params16B
Context16K
Best fitQ4_K_M 11.0 GB
Coding
DE
Qwen2.5 MOE 2X1.5B DeepSeek Uncensored Censored 4B
DeepSeek · DavidAU
Params4.1B
Context128K
Best fitF16 8.9 GB
Reasoning
LL
Meta Llama 3 8B Instruct GGUF Imatrix smashed
Llama · PrunaAI
Params8B
Context8K
Best fitQ8_0 10.1 GB
GGUF
QW
dnotitia_Smoothie Qwen3 4B
Qwen · Alcoft
Params4B
Context40K
Q4_K_M2.4 GB
Reasoning
SA
Saiga_Yandexgpt_8B_Sft_M4_D19_Hf
Saiga · IlyaGusev
Params8B
Context32K
Best fitQ8_0 10.1 GB
GGUF
BO
Borealis Open 12b
Borealis · mradermacher
Params11.8B
Context128K
Best fitQ5_K_S 12.0 GB
GGUF

Other memory sizes

Keep browsing