GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
GE
gemma-2-9b-it
Gemma · legraphista
Params9.24B
Context8K
Best fitIQ3_M 7.8 GB
GL
GLM 4.6V
GLM · mradermacher
Params107B
Context128K
Q4_K_M70.4 GB
QW
Qwen3.5 abliterated MAX AIO
Qwen · prithivMLmods
Params752M
Context256K
Best fitQ8_0 5.4 GB
GR
Granite 3b Code Base 2k
Granite · mradermacher
Params3.48B
Context2K
Best fitQ6_K 6.0 GB
QW
Qwen3.8 27B TQ
Qwen · YTan2000
Params27.3B
Context256K
Smallest0.8 GB
LL
togethercomputer_llama-2-7b-32k-instruct
Llama · TheBloke
Params6.74B
Context32K
Best fitQ3_K_S 7.7 GB
TA
UI TARS 7B DPO
Tars · bartowski
Params7.62B
Context32K
Best fitQ6_K_L 7.5 GB
MI
Mixtral_7Bx2_MoE_13B
Mistral · mradermacher
Params12.9B
Context32K
Best fitQ3_K_M 7.8 GB
MI
Sydney_Pirate_Mistral_7b
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
ME
q-bert_metamath-cybertron-starling
Metamath · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
ST
StarCoder
StarCoder · mradermacher
Params15.5B
Context8K
Best fitIQ3_S 7.6 GB
FL
Sakura FLUX.3 Image Klein PromptUpsampler Uncensored
Flux · webmp3
Params8.95B
Context256K
Best fitQ4_K_M 7.2 GB
HO
Holo4 35B A3B
Holo · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen3 4B toolcalling gguf codex
Qwen · Manojb
Params4B
Context256K
Smallest4.3 GB
QW
Qwen3.5 4B
Qwen · dahara1
Params4.21B
Context256K
Best fitQ8_K_XL 7.5 GB
MI
ANITA NEXT 24B Dolphin Mistral UNCENSORED ITA
Mistral · mradermacher
Params23.6B
Context40K
Best fitIQ1_M 7.6 GB
NE
NVIDIA Nemotron 3.5 Bolt E4B
Nemotron · bartowski
Params7.52B
Context64K
Best fitQ5_K_M 7.9 GB
QW
Swift 1.5 Qwen3.8 27b Heretic
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
DI
digital socrates 7b i1
Digital · mradermacher
Params6.74B
Context4K
Best fitIQ3_M 7.9 GB
GE
Mahou-1.3-gemma2-9B
Gemma · mradermacher
Params9.24B
Context8K
Best fitIQ3_M 7.8 GB
SM
SmolLM 135M
SmolLM · QuantFactory
Params135M
Context2K
Best fitQ2_K 0.8 GB
PE
PentaCoder 9B
Pentacoder · mradermacher
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
QW
QwenPaw Flash 9B heretic MTP
Qwen · SC117
Params456M
Context256K
Best fitQ4_K_M 7.4 GB
FL
Flux2 Klein 9b Uncensored Text Encoder
Flux · wy8edhuh
Params8.19B
Context40K
Best fitQ5_K_M 7.6 GB
GE
Gemma 4 E2B It Uncensored MAX
Gemma · prithivMLmods
Params4.65B
Context128K
Best fitQ8_0 6.0 GB
GR
Granite 20b Code Instruct
Granite · legraphista
Params20B
Context8K
Best fitQ2_K_S 7.9 GB
QW
security 8b
Qwen · bebrws
Params8.2B
Context128K
Best fitQ4_K_M 6.7 GB
GR
Granite 8b Code Base 128k
Granite · RichardErkhov
Params8.05B
Context125K
Best fitQ5_1 7.8 GB
NE
Nemomix v4.0 12B
Nemomix · bartowski
Params12.2B
Context1000K
Best fitQ3_K_M 7.9 GB
EA
EAN 8B i1
Ean · mradermacher
Params8.19B
Context256K
Best fitQ5_K_M 7.6 GB