GGUF models that fit in 12 GB
171517 GGUF language models have at least one quant whose estimated memory use is 12 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Qwen2.5 0.5B Instruct
Qwen · ThomasBaruzier
Params494M
Context32K
Best fitF16 1.6 GB
PH
dolphin-2.9.2-Phi-3-Medium-abliterated
Phi · mradermacher
Params14B
Context4K
Best fitQ5_K_S 11.8 GB
QW
QwQ 32B
Qwen · bartowski
Params32.8B
Context128K
Best fitIQ2_XXS 11.7 GB
LF
LFM2 8B A1B
Lfm · LiquidAI
Params8.34B
Context125K
Best fitQ8_0 11.0 GB
MI
MiMo Negentropy 9B AGSI
Mimo · mradermacher
Params8.95B
Context256K
Best fitQ6_K 8.9 GB
KR
Kronumos Kairos v2
Kronumos · bartowski
Params7.6B
Context32K
Best fitQ8_0 9.1 GB
QW
Gmcoder Bf16
Qwen · emperorofrome
Params8.95B
Context256K
Best fitQ8_0 11.4 GB
XL
xLAM 8x7b R
Xlam · legraphista
Params46.7B
Context32K
Best fitIQ1_S 11.4 GB
LL
Llama 3.3 70B Instruct Abliterated v2
Llama · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
GR
Granite 3.1 8b Instruct
Granite · bartowski
Params8.17B
Context128K
Best fitQ8_0 10.5 GB
WI
Wicked Tutu 26B A4B Final
Wicked · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
PR
Prentis EQ 2
Prentis · mradermacher
Params27B
Context256K
Best fitIQ2_S 12.0 GB
HE
Hermes 4 70B
Hermes · bartowski
Params70.6B
Context128K
Q4_K_M42.5 GB
LL
Hermes-2-Theta-Llama-3-8B
Llama · legraphista
Params8B
Context8K
Best fitQ8_0 10.1 GB
OR
Ornith 1.5 27B A3B Coder MTP
Ornith · ManniX-ITA
Params26.2B
Context256K
Best fitQ2_K_L 11.5 GB
QW
Qwen3.8-27B
Qwen · igroc656
Params27.3B
Context256K
Best fitIQ2_S 11.9 GB
LL
Llama 3.3 8B Instruct
Llama · bartowski
Params8B
Context8K
Best fitQ8_0 10.1 GB
PR
PreciseCoder 27B GRPO 0.5F1
Precisecoder · mradermacher
Params27B
Context256K
Best fitIQ2_S 12.0 GB
CU
CursorCore DS 1.3B SR i1
Cursorcore · mradermacher
Params1.35B
Context16K
Best fitQ6_K 3.3 GB
QW
DeepSeek V4 Pro Qwen3.5 4B MTP Public 20260801 HF
Qwen · Jackrong
Params4.33B
Context256K
Best fitBF16 10.3 GB
PE
Peach 2.0 9B 8k Roleplay
Peach · QuantFactory
Params8.8B
Context16K
Best fitQ8_0 10.7 GB
QW
Qwen3.8 27B Flash MTP
Qwen · SuperLogic
Params461M
Context256K
Smallest6.2 GB
FL
flux model
Flux · licyk
Params11.9B
Context512
Best fitQ5_0 12.0 GB
MI
argilla_capybarahermes-2.5-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ8_0 9.3 GB
TE
Tev1 4B Experimental
Tev · bartowski
Params4.33B
Context256K
Best fitBF16 10.3 GB
QW
Qwen2.5 VL 7B NSFW Caption V3 Abliterated
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ8_0 9.1 GB
DA
datagemma rig 27b it IMat
Datagemma · legraphista
Params27.2B
Context8K
Best fitIQ2_XS 12.0 GB
LE
LeetCodeWizard 13B v1.1a i1
Leetcodewizard · mradermacher
Params13B
Context16K
Best fitIQ2_M 11.7 GB
QW
Qwen2.5 Coder 32B
Qwen · bartowski
Params32.8B
Context32K
Best fitIQ2_XXS 11.7 GB
QW
Mica v0.5 4B
Qwen · sky7350
Params4.21B
Context256K
Best fitBF16 10.0 GB