GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
SM
SmolLM2 135M
SmolLM · prithivMLmods
Params135M
Context8K
Best fitF16 1.0 GB
PH
Phi 4 Mini Instruct
Phi · lmstudio-community
Params3.84B
Context128K
Best fitQ8_0 5.7 GB
TE
Ternary Bonsai 4B
Ternary · prism-ml
Params4B
Context32K
Smallest1.1 GB
OC
Occamy 1.0 APEX I MiniPlus V2.1
Occamy · IsValorum
Params34.7B
Context256K
Best fitQ8_0 3.2 GB
QW
MiMo V2.6 Distill Qwen 9B heretic
Qwen · saidutta69
Params8.95B
Context256K
Best fitQ4_K_M 7.2 GB
HE
0 hero Matter 0.2 7B
Hero · RichardErkhov
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
LL
llama3.1 airoboros3.2 QDT 8B
Llama · bartowski
Params8B
Context128K
Best fitQ5_K_L 7.6 GB
IN
InternVL3_5 2B
Internvl · bartowski
Params2.03B
Context40K
Best fitBF16 5.5 GB
LL
LIama 3.2 test
Llama · bhanuripranathi
Params3.21B
Context128K
Best fitQ4_K_M 3.5 GB
QW
Qwen3.8 Flash Next
Qwen · AesSedai
Params180B
Context256K
Q4_K_M116.1 GB
AR
AREX Turbo
Arex · bartowski
Params4.2B
Context256K
Best fitQ8_0 6.1 GB
BO
Borealis Open 27b
Borealis · mradermacher
Params27B
Context128K
Q4_K_M16.5 GB
QW
Qwen3.8 27B Uncensored
Qwen · shsunshining
Params27.3B
Context256K
Best fitQ8_0 5.8 GB
QW
Huihui Qwen3.6 35B A3B Claude 4.7 Opus Abliterated
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
LF
LFM2.5-8B-A1B-Uncensored-Gaston
Lfm · gaston-parravicini
Params8.47B
Context125K
Best fitQ4_K_M 7.3 GB
QW
Qwen2.5 Coder 1.5B
Qwen · ggml-org
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
MI
MegaBeam Mistral 7B 512k
Mistral · bartowski
Params7.24B
Context512K
Best fitQ6_K_L 7.6 GB
HY
Hy MT2 1.8B 1.25Bit
tencent
Params1.79B
Context256K
Smallest0.5 GB
JE
Jebadiah 9b v2
Jebadiah · bartowski
Params9.2B
Context256K
Best fitQ4_K_L 7.9 GB
TI
TinyLlama 1.1B Chat v0.6
Tinyllama · TinyLlama
Params1.1B
Context2K
Best fitQ4_0 1.3 GB
MI
ehartford_dolphin-2.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
DE
DeepSeek R1 Distill Llama 8B
DeepSeek · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
GE
Gemma 3 270m Qat Q4_0 Unquantized
Gemma · ggml-org
Params268M
Context32K
Best fitQ4_0 0.9 GB
QW
Kataguru Sceptic 35B A3B v2.6
Qwen · kataguru
Params35.2B
Context256K
Best fitQ8_0 3.2 GB
GE
gemma4 coding agent
Gemma · S4MPL3BI4S
Params7.5B
Context128K
Best fitQ4_K_M 7.2 GB
LL
Llm Jp 4 8b Instruct
Llm · mmnga-o
Params8.59B
Context64K
Best fitQ5_K_M 7.7 GB
QW
Qwen2.5-Omni-3B
Qwen · unsloth
Params3.4B
Context32K
Best fitBF16 7.6 GB
NE
Nemotron 3.5 Bolt E4B GSQ RCO
Nemotron · abenzerps
Params7.52B
Context128K
Best fitQ5_K_M 6.5 GB
CA
CAT Thinking 8B
Cat · mmnga-o
Params8.19B
Context40K
Best fitQ5_K_M 7.6 GB
IN
InternLM2
InternLM · legraphista
Params7.74B
Context8K
Best fitQ6_K 7.9 GB