GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
NO
2B_or_not_2B
Not · mradermacher
Params2.5B
Context8K
Best fitQ6_K 2.7 GB
QW
Dolphin3.0 Qwen2.5 1.5B
Qwen · bartowski
Params1.54B
Context128K
Best fitF32 6.9 GB
QW
Qwen3-Vl-4B-Thinking
Qwen · unsloth
Params4B
Context256K
Best fitQ8_K_XL 6.8 GB
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · bartowski
Params8B
Context128K
Best fitQ5_K_L 7.6 GB
QW
Qwen3.8 4B Distilled
Qwen · Ma7ee7
Params4B
Context256K
Best fitQ8_0 6.0 GB
OL
olmOCR 2 7B 1025
OLMo · bartowski
Params7.62B
Context125K
Best fitQ6_K_L 7.5 GB
QW
Qwen3 235B A22B Abliterated
Qwen · mradermacher
Params235B
Context40K
Q4_K_M142.2 GB
QW
Qwen3.5 9B Uncensored HauhauCS Aggressive Q4 K M
Qwen · Manojb
Params8.95B
Context256K
Best fitQ4_K_M 7.2 GB
LA
Laguna S 2.1 Uncensored APEX
Laguna · SC117
Params118B
Context1024K
Best fitBF16 2.9 GB
JE
Jev-Style-0.8B-Decision-v3
Jev · chaoliangUNSW
Params752M
Context256K
Best fitF16 2.4 GB
QW
RuadaptQwen3 4B Instruct
Qwen · RefalMachine
Params4B
Context256K
Best fitQ8_0 6.0 GB
LE
lev
Lev · ggml-org
Params4.2B
Context256K
Best fitQ8_0 6.1 GB
QW
Qwen3.5 9B Uncensored
Qwen · bowmanslayer
Params9.2B
Context256K
Best fitQ4_K_M 7.4 GB
QW
Qwen3.6 35B A3B abliterated v2
Qwen · LMLiutenant
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen3.5 2B Uncensored Abliterix SOM
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
CO
Yi Coder 1.5B Chat
Coder · bartowski
Params1.48B
Context128K
Best fitF16 5.1 GB
DE
DeepSeek R1 Distill Llama 8B Abliterated
DeepSeek · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
NO
North Mini Code 1.0
North · bartowski
Params30.5B
Context488K
Q4_K_M18.7 GB
MI
MiniCPM5 2B Abliterated Uncensored
MiniCPM · mondk
Params2.52B
Context128K
Best fitF16 5.9 GB
QW
Qwen3.8 2B Uncensored
Qwen · atakhadivi
Params1.88B
Context256K
Best fitQ4_K_M 2.2 GB
OC
Occamy 1.0 APEX I MiniPlus V2.1 Abliterated
Occamy · IsValorum
Params34.7B
Context256K
Best fitQ8_0 3.2 GB
GL
GLM 5.3 Flash UNCENSORED NVFP4 DFlash2
GLM · Solstice-AI
Params1.17B
Context1024K
Best fitBF16 3.0 GB
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · lmstudio-community
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
PH
Phi 3 mini 4k instruct update2024 07 03 IMat
Phi · legraphista
Params3.82B
Context4K
Best fitQ8_0 7.8 GB
WI
Wingless_Imp_8B
Wingless · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
MI
MiniCPM OCR Jev
MiniCPM · openbmb
Params1.88B
Context256K
Best fitF16 4.7 GB
LI
vllm-d3-lite
Lite · ggml-org
Params1.01B
Context256K
Best fitBF16 2.9 GB
AE
AesCode 32B
Aescode · bartowski
Params32.8B
Context256K
Q4_K_M20.5 GB
DA
Darwin 27B RSI
Darwin · bartowski
Params27B
Context256K
Q4_K_M17.2 GB
QW
Qwen3-4B-2507-Instruct-Uncensored-HauhauCS-Aggressive
Qwen · HauhauCS
Params4B
Context256K
Best fitQ8_0 6.0 GB