GGUF models that fit in 12 GB

171517 GGUF language models have at least one quant whose estimated memory use is 12 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LF
LFM2.5 8B A1B Hermes Agentic Coder Abliterated v3
Lfm · mradermacher
Params8.47B
Context125K
Best fitQ6_K 9.1 GB
CodingReasoningTool use
LU
Lumimaid Magnum v4 12B
Lumimaid · bartowski
Params12.2B
Context128K
Best fitQ6_K 11.9 GB
GGUF
QW
DPO Trained Qwen2.5 Python Coder 7B
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
Coding
CH
Chaotic Order 24B V5
Chaotic · mradermacher
Params23.6B
Context128K
Best fitIQ3_XS 11.7 GB
GGUF
QW
Qwen3.5 2B Uncensored Zero Refusal SOMA
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
RoleplayVision
QW
Qwen3 Coder Next Uncensored Heretic
Qwen · mradermacher
Params79.7B
Context256K
Q4_K_M48.5 GB
Coding
MI
Tamil-Mistral-7B-v0.1
Mistral · RichardErkhov
Params7.39B
Context32K
Best fitQ8_0 9.4 GB
GGUF
DE
Deacon-34B
Deacon · mradermacher
Params34.4B
Context4K
Best fitIQ2_XXS 11.8 GB
GGUF
YU
Ruach Studio Models v2
Yue · goldhub
Params632M
Context32K
Best fitQ8_0 9.1 GB
Vision
CO
Yi Coder 1.5B
Coder · lmstudio-community
Params1.48B
Context128K
Best fitQ8_0 3.7 GB
Coding
NE
Nemotron 3.5 Embed 8B BF16
Nemotron · W4D
Params7.95B
Context256K
Best fitQ8_0 10.1 GB
EmbeddingVision
RE
REBEL-OpenChat-3.5
Rebel · mradermacher
Params7.24B
Context8K
Best fitQ6_K 7.5 GB
GGUF
MI
Sakura MiniCPM V 4.7 1B Uncensored
MiniCPM · webmp3
Params1.01B
Context256K
Best fitQ8_0 2.0 GB
Vision
MU
Muse Glimmer 30B AZ Focus
Muse · mradermacher
Params27.9B
Context128K
Best fitQ2_K 11.6 GB
ReasoningTool use
QW
Qwen2.5 Coder 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ8_0 9.1 GB
Coding
QW
Qwen3 14B NSFW FDA
Qwen · mradermacher
Params14.8B
Context32K
Best fitQ4_1 11.2 GB
GGUF
QW
Qwen3 1.7B
Qwen · rippertnt
Params2.03B
Context40K
Best fitQ4_K_M 2.7 GB
GGUF
LL
ultravox v0 5 llama 3 1 8b
Llama · ggml-org
Params8B
Context128K
Best fitQ4_K_M 6.5 GB
Vision
GP
SpaceQA Gpt Oss 20b
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
Tool use
QW
Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-BF16
Qwen · Andycurrent
Params4.21B
Context256K
Best fitBF16 10.0 GB
Vision
LF
LFM2 24B A2B
Lfm · bartowski
Params23.8B
Context125K
Best fitQ2_K 11.5 GB
GGUF
CY
Cydonia 22B v1.2
Cydonia · bartowski
Params22.2B
Context32K
Best fitQ3_K_S 12.0 GB
GGUF
QW
Qwen3 0.6B
Qwen · NobodyWho
Params752M
Context32K
Best fitQ4_K_M 1.9 GB
GGUF
PH
Unsloth Phi 3.5 Mini Instruct
Phi · legraphista
Params3.82B
Context128K
Best fitBF16 11.4 GB
GGUF
IN
Internlm2 Math Plus 20b
InternLM · mradermacher
Params19.9B
Context8K
Best fitQ3_K_M 11.8 GB
GGUF
QW
Suri Qwen 3.8 27B Neuro
Qwen · mradermacher
Params27B
Context256K
Best fitIQ2_S 12.0 GB
GGUF
QW
Qwen3 VL 4B Instruct Uncensored
Qwen · mradermacher
Params4B
Context256K
Best fitF16 9.8 GB
Vision
QW
FlashModel Qwen 3.6 35B A3B Maxed
Qwen · flashback2k
Params35.5B
Context256K
Q4_K_M21.7 GB
ReasoningTool useVision
OP
OpenELM-3B-Instruct
Openelm · SanctumAI
Params3.04B
Context2K
Best fitF16 7.3 GB
GGUF
KN
KnowLine 9B Gen1
Knowline · mradermacher
Params9.2B
Context256K
Best fitQ8_0 11.4 GB
ReasoningTool useVision

Other memory sizes

Keep browsing