GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QE
qev 0.8b
Qev · labqoat
Params752M
Context256K
Best fitBF16 2.4 GB
LL
llamas-community_llamaguard-7b
Llama · TheBloke
Params6.74B
Context2K
Best fitQ3_K_S 7.7 GB
LL
tiny random LlamaForCausalLM
Llama · aladar
Params1M
Context2K
Smallest0.0 GB
QW
OpenChat 3.5 7B Qwen v2.0
Qwen · mradermacher
Params7.24B
Context8K
Best fitQ6_K 7.5 GB
MI
Ministral 3 8B Reasoning 2512 Esper3.1
Mistral · mradermacher
Params8.49B
Context256K
Best fitQ5_K_M 7.7 GB
QW
Qwopus3.8 27B Flash V2 Apostate Uncensored
Qwopus · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
LL
Meta Llama 3 8B
Llama · QuantFactory
Params8B
Context8K
Best fitQ5_1 7.6 GB
PH
Phi 3 Mini 128k Instruct
Phi · second-state
Params3.82B
Context128K
Best fitQ8_0 7.8 GB
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
GE
Gemma 3 1b It Qat
Gemma · bartowski
Params1B
Context32K
Best fitBF16 2.7 GB
LM
LMCocktail 10.7B v1
Lmcocktail · TheBloke
Params10.7B
Context4K
Best fitQ3_K_L 7.8 GB
CL
Claire 7B 0.1
Claire · TheBloke
Params7.22B
Context2K
Best fitQ6_K 7.6 GB
MI
Hermes-2-Pro-Mistral-7B-infinity
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
RA
al rasheed ai
Rasheed · alpha-coder-great
Params3B
Context128K
Best fitQ4_K_M 3.5 GB
QW
QwenDGAF V3 4B
Qwen · hecan3d
Params4B
Context40K
Best fitQ8_0 6.0 GB
JA
Javi0.3 2B
Javi · mradermacher
Params1.94B
Context256K
Best fitQ6_K 2.5 GB
LL
gpt 4o distil Llama 3.3 70B Instruct PaperWitch heresy i1
Llama · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
IN
OpenChat 3.5 7B InternLM v2.0
InternLM · mradermacher
Params7.24B
Context8K
Best fitQ6_K 7.5 GB
QW
FlashModel Qwen 3.6 35B A3B Maxed
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
GE
medgap gemma 4 12b
Gemma · protocolsyncllc
Params11.9B
Context256K
Best fitQ8_0 2.0 GB
XO
XORTRON AgentRICO
Xortron · bartowski
Params27.3B
Context256K
Q4_K_M17.4 GB
MI
Mistral Syndicate 7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
MI
MiniCPM4.1 8B
MiniCPM · openbmb
Params8.2B
Context64K
Best fitQ4_K_M 5.7 GB
NE
Nehuronion Dvt 9b
Nehuronion · mradermacher
Params9.24B
Context8K
Best fitIQ3_M 7.8 GB
LL
models--Groq--Llama-3-Groq-8B-Tool-Use
Llama · MaziyarPanahi
Params8B
Context8K
Best fitQ5_K_M 7.3 GB
QW
Qwen3.8-27B-DFlash2
Qwen · magnitudedev
Params1.92B
Context256K
Best fitQ8_0 2.7 GB
OR
Ornith 1.5 122B
Ornith · mradermacher
Params125B
Context256K
Q4_K_M75.8 GB
GR
Granite 3b Code Instruct 128k
Granite · RichardErkhov
Params3.48B
Context125K
Best fitQ8_0 6.9 GB
QW
Qwen3.6 35B A3B Uncensored Genesis Final Safetensors
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB