GGUF models that fit in 16 GB
182247 GGUF language models have at least one quant whose estimated memory use is 16 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
SM
SmolLM2 1.7B Instruct
SmolLM · bartowski
Params1.71B
Context8K
Best fitF16 5.5 GB
QW
Qwen3.5-9B
Qwen · weareapexcreators
Params9.2B
Context256K
Best fitQ5_K_XL 8.5 GB
QW
Qwen2.5 7b Ins v3
Qwen · bartowski
Params7.62B
Context32K
Best fitQ8_0 9.1 GB
QW
OpenYourMind Qwen3.6 35B A3B Kuato DPO Abliterated Uncensored
Qwen · mradermacher
Params34.7B
Context256K
Best fitQ2_K 14.1 GB
OR
Ornith 1.0 35B
Ornith · weareapexcreators
Params34.7B
Context1024K
Best fitQ2_K 14.1 GB
DE
deepseek-ai_deepseek-coder-1.3b-instruct
DeepSeek · TheBloke
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
LL
Llama 3.1 8B Stheno v3.4
Llama · bartowski
Params8B
Context128K
Best fitQ8_0 10.1 GB
OR
Ornith 1.5 9B
Ornith · abenzerps
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
MI
MinerU2.5 Pro 2605 1.2B
Mineru · jinzhenj
Params494M
Context8K
Best fitQ8_0 1.1 GB
TR
Translategemma 12b It
Translategemma · bullerwins
Params11.8B
Context128K
Best fitQ6_K 13.4 GB
LL
Anubis Mini 8B v1
Llama · TheDrummer
Params8B
Context128K
Best fitQ8_0 10.1 GB
WA
WaifuGemma4 26b a4b v1 i1
Waifugemma · hiwaifu-research
Params25.2B
Context256K
Q4_K_M16.8 GB
QW
speakoflow mini
Qwen · SpeakoFlow
Params773M
Context256K
Best fitBF16 2.5 GB
QW
Qwen2.5-Omni-7B
Qwen · unsloth
Params7.62B
Context32K
Best fitQ8_K_XL 11.2 GB
OR
Ornith 1.0 9B MTP LocalAI
Ornith · weareapexcreators
Params9.2B
Context256K
Best fitQ4_K_M 7.4 GB
MI
vllm-d3-mini
Mini · ggml-org
Params4.84B
Context256K
Best fitBF16 11.3 GB
QW
Qwen2.5 3B Instruct Abliterated
Qwen · mradermacher
Params3.4B
Context32K
Best fitF16 7.6 GB
GE
Gemma 4 26B A4B StyleTune V2
Gemma · mradermacher
Params26B
Context256K
Q4_K_M17.2 GB
QW
Qwen3 VL 8B Instruct Abliterated v2.0
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ8_0 10.4 GB
QW
Qwen3.5 9B Claude 4.6 HighIQ INSTRUCT HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ8_0 11.1 GB
QW
Qwen3.8 27B Omnimerge v6
Qwen · ManniX-ITA
Params27.3B
Context256K
Best fitIQ3_M 15.4 GB
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ8_0 9.1 GB
QW
Mica v0.1 4B
Qwen · sky7350
Params4.84B
Context256K
Best fitBF16 11.3 GB
UN
Undi95 Emerhyst 20B
Undi · RichardErkhov
Params920M
Context—
Best fitQ2_K 8.8 GB
MI
Dolphin Mistral 24B Venice Edition
Mistral · douganger
Params23.6B
Context128K
Best fitIQ4_XS 14.6 GB
MI
Ministral-3-14B-Instruct-2512
Mistral · unsloth
Params13.5B
Context256K
Best fitQ6_K_XL 14.0 GB
BG
bge-small-zh-v1.5
BGE · CompendiumLabs
Params24M
Context512
Best fitF32 0.7 GB
QW
Qwen3.5-35B-A3B
Qwen · unsloth
Params35.5B
Context256K
Best fitIQ3_XXS 15.3 GB
QW
Ingot XIV 27B Q2
Qwen · dendriteholdings
Params26.9B
Context256K
Smallest5.9 GB
OP
opendecider small
Opendecider · manjunathshiva
Params4.02B
Context256K
Best fitQ8_0 6.0 GB