GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
qwen3loop 0.6b sft deep supervision v1
Qwen · Lordnyx
Params596M
Context40K
Best fitF16 4.5 GB
DE
deepseek r1
DeepSeek · cortexso
Params70.6B
Context128K
Best fitQ3_K_S 7.4 GB
CA
CarbonVillain en 10.7B v4
Carbonvillain · TheBloke
Params10.7B
Context4K
Best fitQ3_K_L 7.8 GB
LL
Llama 3.1 WhiteRabbitNeo 2 8B
Llama · bartowski
Params8B
Context128K
Best fitQ5_K_L 7.6 GB
ZE
Zeta 2.1
Zeta · mradermacher
Params8.25B
Context32K
Best fitQ5_K_M 7.5 GB
BR
Breeze-13B-32k-Instruct-v1_0
Breeze · mradermacher
Params12.7B
Context32K
Best fitQ3_K_S 8.0 GB
QW
Qwen3 14B Abliterated
Qwen · mradermacher
Params14.8B
Context40K
Best fitIQ3_XXS 7.8 GB
QW
Huihui Qwen3.5 9B Claude 4.6 Opus Abliterated Heretic
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
OP
openPangu Embedded 7B V1.1
Openpangu · mradermacher
Params8.03B
Context32K
Best fitQ5_K_M 7.4 GB
TH
TherapyBeagle 11B v1 i1
Therapybeagle · mradermacher
Params10.7B
Context32K
Best fitIQ4_XS 7.9 GB
FA
Falcon H1 7B Instruct
Falcon · tiiuae
Params7.59B
Context256K
Best fitQ6_K 7.1 GB
EC
EchoVLM_v2
Echovlm · mradermacher
Params8.95B
Context256K
Best fitQ5_K_S 7.9 GB
SY
maywell_synatra-7b-v0.3-base
Synatra · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
QW
GPT neo Custom 7B
Qwen · Szymon-Wachelko
Params7.6B
Context32K
Best fitQ6_K 7.2 GB
YI
EVA Yi 1.5 9B 32K V1
Yi · bartowski
Params8.83B
Context32K
Best fitQ5_K_L 7.7 GB
IN
Internlm2 Math Plus 7b
InternLM · mradermacher
Params7.74B
Context8K
Best fitQ6_K 7.9 GB
ER
ERNIE 4.5 21B A3B PT
Ernie · bartowski
Params21.8B
Context128K
Best fitIQ2_XS 7.3 GB
SY
migtissera_synthia-7b-v1.3
Synthia · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
GE
gemma 4 E4B it heretic
Gemma · igorls
Params7.5B
Context128K
Best fitQ5_K_M 7.7 GB
FA
MN 12B Faun RP RU
Faun · limloop
Params12.2B
Context1000K
Best fitQ3_K_M 7.9 GB
QW
Qwen3.5 40B Claude 4.6 Opus Deckard Heretic Uncensored Thinking
Qwen · FrosT-LyRo
Params39.1B
Context256K
Q4_K_M23.9 GB
QW
Qwen3-Vl-2B-Thinking-1M
Qwen · unsloth
Params1.72B
Context977K
Best fitBF16 4.9 GB
CA
CarbonVillain En 10.7B v4
Carbonvillain · mradermacher
Params10.7B
Context4K
Best fitIQ4_XS 7.9 GB
LF
LMF 2.5 2B Code
Lfm · bunnycore
Params2.7B
Context128K
Best fitQ5_K_M 4.5 GB
LL
Collaiborator-MEDLLM-Llama-3-8B-v2
Llama · mradermacher
Params8B
Context8K
Best fitQ5_K_M 7.3 GB
QW
Qwen3.8 9B Function Calling xLAM Unsloth
Qwen · ermiaazarkhalili
Params9.2B
Context256K
Best fitQ4_K_M 7.4 GB
TH
Thaki ASR KO Dialects v1 i1
Thaki · mradermacher
Params1.72B
Context64K
Best fitQ6_K 2.9 GB
NO
nomic ai nomic embed code
Nomic · bartowski
Params7.1B
Context32K
Best fitQ6_K_L 6.9 GB
AM
AmberChat
Amberchat · mradermacher
Params6.74B
Context2K
Best fitIQ3_M 7.9 GB
EK
Ekbasis Skopos 27B
Ekbasis · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB