GGUF models that fit in 16 GB
182247 GGUF language models have at least one quant whose estimated memory use is 16 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3.8 Whittle 16B
Qwen · logic65
Params16.3B
Context256K
Best fitQ4_K_M 12.1 GB
OP
openPangu 2.0 Flash
Openpangu · mrexodia
Params91.6B
Context512K
Best fitQ8_0 10.5 GB
JA
AI21 Jamba Reasoning 3B
Jamba · bartowski
Params3.2B
Context256K
Best fitBF16 9.3 GB
SA
Salience 27B R5
Salience · bartowski
Params27.3B
Context1024K
Best fitIQ3_XS 16.0 GB
NE
NeuralMona MoE 4x7B i1
Neuralmona · mradermacher
Params24.2B
Context32K
Best fitQ4_K_S 15.5 GB
KA
KAI 7B Instruct v0.1
Kai · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
PS
Psyonic-Cetacean-full-precision
Psyonic · mradermacher
Params20B
Context4K
Best fitIQ1_M 15.7 GB
TE
Ternary Bonsai 2 27B Abliterated swift cap PQ2 0 MTP
Ternary · hotdogs
Params27.3B
Context256K
Smallest7.6 GB
VE
Veritas 8B Fact Checker Non Thinking 1.0
Veritas · mradermacher
Params8.19B
Context40K
Best fitQ6_K 8.4 GB
LL
model
Llama · ChatGpt1
Params8B
Context8K
Best fitQ8_0 10.1 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · CISCai
Params7.62B
Context32K
Best fitIQ4_XS 5.2 GB
HU
HumanlikeRP
Humanlikerp · mradermacher
Params8.83B
Context12K
Best fitQ6_K 8.6 GB
BE
Behemoth X 123B v2.2
Behemoth · bartowski
Params123B
Context128K
Q4_K_M76.3 GB
MI
Trinity 2 Codestral 22B v0.2
Mistral · bartowski
Params22.2B
Context32K
Best fitQ4_K_L 15.9 GB
NE
Nex N2.5 Mini
Nex · IsValorum
Params34.7B
Context256K
Smallest14.8 GB
QW
Qwen3.5 4B EmperoAI Qwen3.8 Distill Heretic Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
MI
paixai_astrid-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ8_0 9.3 GB
LL
LlamaScientific 8B
Llama · mradermacher
Params8.03B
Context128K
Best fitQ6_K 8.2 GB
RO
rocket 3B
Rocket · TheBloke
Params2.8B
Context4K
Best fitQ8_0 6.2 GB
JI
Jina Reranker v3.5
Jina · jinaai
Params596M
Context128K
Best fitBF16 2.6 GB
QW
Qwen3.5 4B DFlash
Qwen · magnitudedev
Params634M
Context40K
Best fitQ8_0 1.4 GB
TI
Tifa Deepsex 14b CoT GGUF Q4
Tifa · ValueFX9507
Params14.8B
Context128K
Best fitQ4_K_M 11.1 GB
MI
Ministral-3-3B-Reasoning-2512
Mistral · Mungert
Params3.43B
Context256K
Best fitBF16 8.2 GB
QW
Agrillm Qwen3 30B A3B
Qwen · mradermacher
Params30.5B
Context256K
Best fitQ3_K_M 16.0 GB
QW
Qwen2.5 VL 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ8_0 9.1 GB
SM
KrynexAI 1.4
SmolLM · KrynexLabs
Params135M
Context8K
Best fitF16 1.0 GB
NO
Nous Hermes 13B Code
Nous · TheBloke
Params13B
Context4K
Best fitQ4_K_M 15.1 GB
QW
Co Rewarding I Qwen3 8B Base MATH
Qwen · mradermacher
Params8.19B
Context32K
Best fitQ6_K 8.4 GB
SE
Sexpedition MS3.2 24B
Sexpedition · mradermacher
Params23.6B
Context128K
Best fitQ4_K_S 15.4 GB
FA
Falcon H1 34B Instruct
Falcon · tiiuae
Params33.6B
Context256K
Best fitIQ3_XXS 15.1 GB