GGUF models that fit in 8 GB

151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5 9B EmperoAI Qwen3.8 Distill Heretic CyberStrike
Qwen · Aapolon
Params9.2B
Context256K
Best fitQ4_K_M 7.4 GB
Vision
QW
Nyx RP 9B Instruct 2608 v1
Qwen · Indexnusrefather
Params9.2B
Context256K
Best fitQ4_K_M 7.4 GB
Roleplay
MI
maywell_pivot-10.7b-mistral-v0.2-rp
Mistral · TheBloke
Params10.7B
Context32K
Best fitQ3_K_L 7.8 GB
Roleplay
LL
saraprice llama2 7B backdoor DEPLOYMENT
Llama · RichardErkhov
Params6.74B
Context4K
Best fitIQ3_M 7.9 GB
GGUF
MI
Hermes-2-Pro-Mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
Tool use
QW
Qwen3.8 27B OBLITERATED Mythos Class Agentic
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
ReasoningTool use
GP
Gpt Oss 20b Unsloth Puzzle V2
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
Reasoning
TR
TranslatePsy AfriSLM 4B
Translatepsy · mradermacher
Params4.84B
Context256K
Best fitQ6_K 5.6 GB
ReasoningTool use
MI
lemonilia_ashhlimarp-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
GGUF
ON
OneJev 0.8B
Onejev · bartowski
Params1.01B
Context256K
Best fitBF16 2.9 GB
Vision
HO
K2 Horizon MoVA 36B A4B
Horizon · ngquocvinh
Params37.4B
Context512K
Best fitQ1_0 7.9 GB
GGUF
LL
Llama Leonida
Llama · mradermacher
Params8.2B
Context128K
Best fitQ5_K_M 7.4 GB
Tool use
ME
models--automerger--T3qm7Mergerix-7B
Mergerix · MaziyarPanahi
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
GGUF
LF
LFM2.5 8B A1B heretic
Lfm · saidutta69
Params8.47B
Context125K
Best fitQ4_K_M 7.3 GB
ReasoningRoleplayTool use
ST
StartLux-Decision-v1.1-0.8B-Instant
Startlux · startlux-models
Params752M
Context256K
Best fitQ8_0 1.7 GB
Vision
AN
Andromeda 8B Coder i1
Andromeda · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
CodingTool use
YA
models--automerger--Experiment28Yam-7B
Yam · MaziyarPanahi
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
GGUF
ST
stablelm zephyr 3b
Stablelm · TheBloke
Params2.8B
Context4K
Best fitQ8_0 6.2 GB
GGUF
EV
EAN 7B i1
Evan · mradermacher
Params7.62B
Context125K
Best fitQ6_K 7.2 GB
Reasoning
SU
SummLlama3 8B i1
Summllama · mradermacher
Params8.03B
Context8K
Best fitQ5_K_M 7.3 GB
GGUF
QW
Trohrbaugh Qwen3.5 122B A10B Heretic
Qwen · mradermacher
Params122B
Context256K
Q4_K_M74.2 GB
ReasoningTool use
QW
Qwen3-4B-128K
Qwen · unsloth
Params4B
Context128K
Best fitQ8_K_XL 6.8 GB
GGUF
TI
Tiamat-7b-1.1-DPO
Tiamat · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
GGUF
LL
Llama-3-Patronus-Lynx-8B-Instruct
Llama · mradermacher
Params8B
Context8K
Best fitQ5_K_M 7.3 GB
GGUF
RO
ROMA 7B
Roma · mradermacher
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
GGUF
LL
meta-llama-7b
Llama · TheBloke
Params6.74B
Context2K
Best fitQ3_K_S 7.7 GB
GGUF
GR
GRM 2.6 Plus 0628
Grm · bartowski
Params27.3B
Context256K
Q4_K_M17.8 GB
Vision
SM
SmolVLM Instruct
Smolvlm · ggml-org
Params1.81B
Context16K
Best fitF16 5.7 GB
Vision
KA
KAI 7B v0.1
Kai · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
Coding
LF
LFM2 1.2B Tool
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
Tool use

Other memory sizes

Keep browsing