GGUF models that fit in 12 GB
171517 GGUF language models have at least one quant whose estimated memory use is 12 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
SA
Sakura SOLAR Instruct
Sakura · TheBloke
Params10.7B
Context4K
Best fitQ6_K 10.9 GB
LL
Llama3.1 8B ShiningValiant2
Llama · bartowski
Params8B
Context128K
Best fitQ8_0 10.1 GB
MI
MiniCPM V 4.7 1B
MiniCPM · NANI-Nithin
Params1.01B
Context256K
Best fitBF16 2.9 GB
AV
Avrik Lite 4B
Avrik · mradermacher
Params7.52B
Context128K
Best fitQ6_K 8.1 GB
GE
GELab Zero 4B Preview
Gelab · bartowski
Params4B
Context256K
Best fitBF16 9.8 GB
MA
Magicoder S DS 6.7B
Magicoder · TheBloke
Params6.74B
Context16K
Best fitQ8_0 12.0 GB
QW
MN 2407 DSK QwQify v0.1 12B
Qwen · bartowski
Params12.2B
Context128K
Best fitQ6_K 11.9 GB
CY
Cydonia 24B v4.3 Absolute Heresy
Cydonia · mradermacher
Params23.6B
Context128K
Best fitQ2_K 10.7 GB
QW
Qwen3.5 9b Claude 4.8 Opus reasoning
Qwen · TheCidSama
Params9.2B
Context256K
Smallest5.8 GB
DO
weyaxi_dolphin2.1-openorca-7b
Dolphin · TheBloke
Params7.24B
Context32K
Best fitQ8_0 9.3 GB
LL
Llama-2-7b-WikiChat-fused
Llama · RichardErkhov
Params6.74B
Context4K
Best fitQ8_0 12.0 GB
LL
CodeLlama 7b Instruct Hf
Llama · mradermacher
Params6.74B
Context16K
Best fitQ6_K 10.3 GB
QW
Video Companions
Qwen · off-grid-ai
Params4.02B
Context256K
Best fitQ4_K_M 5.4 GB
ST
L3 Stheno Maid Blackroot Grand HORROR 16B
Stheno · DavidAU
Params16.5B
Context8K
Best fitIQ4_XS 11.9 GB
LL
Llama3Ternary 10b
Llama · CMSManhattan
Params9.64B
Context8K
Best fitTQ2_0 5.9 GB
UN
Unlimited OCR 2
Unlimited · AtomicChat
Params2.94B
Context128K
Best fitBF16 6.9 GB
PH
Phi 4 Reasoning
Phi · bartowski
Params14.7B
Context32K
Best fitQ4_K_L 11.6 GB
LL
rombos Llama3 8B Instruct Replete Adapted i1
Llama · mradermacher
Params8B
Context8K
Best fitQ6_K 8.2 GB
QW
Qwen2.5 Coder 7B
Qwen · QuantFactory
Params7.6B
Context32K
Best fitQ8_0 9.1 GB
MI
nondzu_mistral-7b-code-16k-qlora
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ8_0 9.3 GB
YI
Faro Yi 34B i1
Yi · mradermacher
Params34.4B
Context195K
Best fitIQ2_XXS 11.8 GB
WI
WizardCoder Python 13B V1.0 i1
Wizardcoder · mradermacher
Params13B
Context16K
Best fitIQ2_M 11.7 GB
OR
Ornith 1.5 9B Heretic
Ornith · mradermacher
Params8.95B
Context256K
Best fitQ6_K 8.9 GB
CH
Chocolatine 14B Instruct DPO v1.2 i1
Chocolatine · mradermacher
Params14B
Context4K
Best fitQ5_K_S 11.8 GB
KE
Kev-4B Artemis 2.5GB
Kev · turintech
Params4.21B
Context256K
Best fitQ4_K_M 4.1 GB
IN
INTELLECT 3
Intellect · bartowski
Params107B
Context128K
Q4_K_M71.4 GB
EM
embeddinggemma-2
Embeddinggemma · AtomicChat
Params271M
Context256K
Best fitBF16 2.7 GB
EM
Embeddinggemma 300M Qat Q4_0 Unquantized
Embeddinggemma · ggml-org
Params303M
Context2K
Best fitQ4_0 1.0 GB
QW
Qwen3.5 4B Heretic 2xMerge DavidAU
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
LL
LosslessMegaCoder Llama2 7B Mini
Llama · TheBloke
Params6.74B
Context4K
Best fitQ8_0 12.0 GB