GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
GE
Ablation 187 Gemma3.Webnovels.Test.Gemma 3 4b
Gemma · mradermacher
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
GR
Granite 4.0 Micro
Granite · ibm-granite
Params3.4B
Context128K
Best fitQ8_0 7.5 GB
JI
jinen-v1-xsmall-hf
Jinen · togatogah
Params36M
Context1K
Best fitF16 0.7 GB
PH
phi3.5 gutenberg 4B
Phi · bartowski
Params3.82B
Context128K
Best fitQ8_0 7.8 GB
SE
Sexpedition MS3.2 24B
Sexpedition · mradermacher
Params23.6B
Context128K
Best fitIQ1_M 7.6 GB
CO
Confucius4 R2T2
Confucius · netease-youdao
Params1.72B
Context64K
Best fitF16 4.9 GB
TE
Ternary Bonsai 1.7B
Ternary · prism-ml
Params1.72B
Context32K
Best fitF16 4.9 GB
QW
Huihui Qwen3 8B Abliterated v2
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ5_K_M 7.6 GB
DA
L3 Dark Planet 8B
Dark · DavidAU
Params8B
Context8K
Best fitQ5_K_M 7.5 GB
PH
phi 4
Phi · unsloth
Params14.7B
Context16K
Best fitQ2_K_L 7.9 GB
ST
stablelm 2 zephyr 1 6b
Stablelm · stabilityai
Params1.64B
Context4K
Smallest1.0 GB
QW
Qwen2.5 VL 7B Abliterated Caption It
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ6_K 7.2 GB
LL
Hermes 3 Llama 3.2 3b Base Fft Chatml Rerun 8xh100 E4 2024 11 24 4k Steps
Llama · NousResearch
Params3.21B
Context128K
Best fitQ8_0 4.9 GB
MI
dolphin 2.9.3 mistral nemo 12b
Mistral · dphn
Params12.2B
Context1000K
Best fitQ3_K_M 7.9 GB
QW
Qwen3 VL Reranker 8B
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ5_K_M 7.6 GB
OR
Ornith1.0 9B 3MPER0RR Abliterated
Ornith · mradermacher
Params8.95B
Context256K
Best fitQ5_K_S 7.9 GB
ON
OneJev 4B
Onejev · mradermacher
Params4.84B
Context256K
Best fitQ8_0 6.7 GB
LF
LFM2.5 VL 3B Uncensored
Lfm · SC117
Params2.7B
Context125K
Best fitBF16 7.9 GB
PH
Phi 4 Mini Reasoning
Phi · lmstudio-community
Params3.84B
Context128K
Best fitQ8_0 5.7 GB
ST
berkeley-nest_starling-lm-7b-alpha
Starling · TheBloke
Params7.24B
Context8K
Best fitQ6_K 7.5 GB
QW
Qwen3 14B ARPO DeepSearch
Qwen · mradermacher
Params14.8B
Context40K
Best fitIQ3_XXS 7.8 GB
JE
JEV 9B
Jev · prithivMLmods
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
QW
Qwen3.8-27B-DFlash2-ternary-bonsai2-r3
Qwen · naklitechie
Params1.92B
Context256K
Best fitQ4_K_M 1.8 GB
QW
Qwen3.5 4B Heretic
Qwen · darrellbest
Params4.33B
Context256K
Best fitQ8_0 6.2 GB
MY
Mythos v2 9b
Mythos · fableforge-ai
Params9.2B
Context256K
Best fitQ4_1 7.6 GB
HO
Holo 3.1 4B
Holo · mradermacher
Params4.84B
Context256K
Best fitQ8_0 6.7 GB
TO
tokyotech llm Swallow MS 7b instruct v0.1
Tokyotech · mmnga
Params7.33B
Context4K
Best fitQ6_K 7.6 GB
QW
qwen2-7b-instruct
Qwen · Qwen
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
QW
Qwen3.5 9B Cold Fusion GAIN v1.0 Uncensored Heretic NEO MAX Imatrix
Qwen · DavidAU
Params8.95B
Context256K
Best fitIQ4_XS 8.0 GB
QW
Qwen3.5 0.8B GGUF uncensored opus distill
Qwen · amkkk
Params752M
Context256K
Best fitQ4_K_M 1.4 GB