GGUF models that fit in 16 GB
182247 GGUF language models have at least one quant whose estimated memory use is 16 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
SH
Shadow-Siren-14B-v2-Pruned
Shadow · mradermacher
Params13.8B
Context256K
Best fitQ3_K_M 15.9 GB
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Best fitQ6_K 7.2 GB
IN
InternVL3_5 38B
Internvl · bartowski
Params32.8B
Context40K
Best fitQ2_K_L 15.8 GB
PH
Phi 3 Medium 4k Instruct
Phi · mradermacher
Params14B
Context4K
Best fitQ6_K 13.6 GB
IN
InternVL3_5 38B
Internvl · mradermacher
Params32.8B
Context40K
Best fitQ2_K 15.0 GB
QW
Qwen2.5 32B
Qwen · mradermacher
Params32.8B
Context128K
Best fitIQ3_XXS 15.5 GB
LF
LFM2.5 1.2B JP 202606
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
QW
Qwen2.5 Coder 7B Instruct Abliterated
Qwen · nyan-ame
Params7.6B
Context32K
Best fitQ8_0 9.1 GB
QW
Qwen3.8-27B
Qwen · LxBarn
Params27.3B
Context256K
Best fitQ3_K_XL 16.0 GB
DE
Deepseeker Kunou Qwen2.5 14b i1
DeepSeek · mradermacher
Params14.8B
Context128K
Best fitQ6_K 14.2 GB
LF
LFM2.5 24B A2B Uncensored
Lfm · pottokao
Params24B
Context128K
Best fitIQ4_XS 16.0 GB
QW
Qwopus3.5 27B v3 Abliterated
Qwopus · mradermacher
Params27B
Context256K
Best fitQ3_K_M 15.9 GB
ZW
ZwZ2 4B
Zwz · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
QW
Qwen2.5 Coder 3B Instruct
Qwen · bocalan
Params3.09B
Context32K
Best fitQ4_K_M 2.7 GB
LL
Hermes 3 Llama 3.1 8B Lorablated
Llama · bartowski
Params8B
Context128K
Best fitQ8_0 10.1 GB
MI
Pixtral
Mistral · ggml-org
Params12.2B
Context1000K
Best fitQ8_0 14.9 GB
LL
Meta-Llama-3-8B-Instruct-abliterated-v3
Llama · legraphista
Params8B
Context8K
Best fitQ8_0 10.1 GB
MI
kolibri-vikhr-mistral-0427
Mistral · Vikhrmodels
Params7.6B
Context32K
Best fitQ8_0 9.7 GB
MI
mlabonne_neuralhermes-2.5-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ8_0 9.3 GB
MI
MiniCPM V 4.7 35B A3B
MiniCPM · bartowski
Params34.7B
Context256K
Best fitQ2_K 15.0 GB
TR
Trida2.0 4B
Trida · mradermacher
Params4.84B
Context256K
Best fitQ6_K 5.6 GB
EU
EuroLLM 9B Instruct
Eurollm · bartowski
Params9.15B
Context4K
Best fitQ8_0 11.6 GB
QW
Qwen2.5 32B AGI
Qwen · mradermacher
Params32.8B
Context32K
Best fitIQ3_XXS 15.5 GB
LL
LLaMA2 13B Tiefighter i1
Llama · mradermacher
Params13B
Context4K
Best fitQ4_K_M 15.3 GB
WI
ehartford_wizardlm-7b-v1.0-uncensored
Wizardlm · TheBloke
Params6.74B
Context2K
Best fitQ8_0 12.0 GB
NE
NeoHorse 1 4B
Neohorse · mradermacher
Params4.21B
Context256K
Best fitF16 10.0 GB
PI
piano-medley-7b
Piano · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
AM
AMALIA 9B 0626 SFT
Amalia · NoxNotreve
Params9.15B
Context32K
Best fitQ8_0 11.6 GB
GR
Granite 34b Code Instruct 8k
Granite · RichardErkhov
Params33.7B
Context8K
Best fitIQ3_S 15.7 GB
RW
rwitz_go-bruins-v2
Rwitz · TheBloke
Params7.24B
Context32K
Best fitQ8_0 9.3 GB