GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
PH
Phi 4 Mini Instruct
Phi · MaziyarPanahi
Params3.84B
Context128K
Best fitQ8_0 5.7 GB
LL
Dolphin 3.0 Llama 3.1 8B
Llama · dphn
Params8B
Context128K
Best fitQ5_1 7.6 GB
QW
Qwen3 Embedding 8B
Qwen · Qwen
Params7.57B
Context40K
Best fitQ6_K 7.9 GB
GE
Winnow E4B
Gemma · EldanRing
Params78M
Context128K
Best fitBF16 0.8 GB
LL
chronicle narrator
Llama · GWsuryaYT
Params8B
Context128K
Best fitQ4_K_M 6.5 GB
GL
GLM OCR
GLM · ggml-org
Params891M
Context128K
Best fitF16 2.9 GB
QW
Models Qwen Qwen2.5 7B Instruct
Qwen · MaziyarPanahi
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
LL
Llama-3.2-1B-Instruct
Llama · unsloth
Params1.24B
Context128K
Best fitF16 3.2 GB
SO
Models Upstage Solar Pro Preview Instruct
Solar · MaziyarPanahi
Params22B
Context4K
Best fitIQ1_S 8.0 GB
DE
Deepseek-R1-Distill-Llama-8B
DeepSeek · unsloth
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
D1
d1 3B
LiquidAI
Params2.7B
Context32K
Best fitBF16 7.9 GB
OR
Ornith 1.5 35B A3B
Ornith · bartowski
Params35.5B
Context256K
Q4_K_M21.9 GB
CO
Models 01 Ai Yi Coder 9B Chat
Coder · MaziyarPanahi
Params8.83B
Context128K
Best fitQ5_K_M 7.6 GB
QW
Models Qwen Qwen2.5 1.5B Instruct
Qwen · MaziyarPanahi
Params1.54B
Context32K
Q4_K_M1.0 GB
LL
Meta Llama 3 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Best fitQ5_1 7.6 GB
CO
Models 01 Ai Yi Coder 1.5B Chat
Coder · MaziyarPanahi
Params1.48B
Context128K
Q4_K_M1.0 GB
QW
Qwen2.5 VL 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ6_K 7.2 GB
MI
Models Mistralai Ministral 3 3B Reasoning 2512
Mistral · MaziyarPanahi
Params3.43B
Context256K
Best fitQ6_K 4.2 GB
GE
Models Google Gemma 2 2b It
Gemma · MaziyarPanahi
Params2.61B
Context8K
Q4_K_M1.7 GB
CL
Clef Flash
Clef · bartowski
Params9.1B
Context256K
Best fitQ4_K_M 7.6 GB
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Best fitQ4_1 1.8 GB
MI
MiniCPM V 4 5
MiniCPM · openbmb
Params8.2B
Context40K
Best fitQ5_1 7.9 GB
GE
Gemma 4 E4B It OBLITERATED v3.1
Gemma · OBLITERATUS
Params7.5B
Context128K
Best fitQ5_K_M 7.7 GB
WI
models--microsoft--WizardLM-2-7B
Wizardlm · MaziyarPanahi
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
GE
Models Google Gemma 3 1b It
Gemma · MaziyarPanahi
Params1B
Context32K
Q4_K_M0.8 GB
BE
BeaverAI NeMoist 21B v1a
Beaverai · RichardErkhov
Params21B
Context—
Best fitQ3_K_S 7.4 GB
QW
Qwen Image 2.1 Uncensored
Qwen · 0xSojalSec
Params7.12B
Context—
Best fitQ6_K 7.1 GB
QW
Qwen3.8 27B Heretic JP Roleplay NSFW DanbooruTags
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
QW
Qwen3 TTS 12Hz 1.7B Base
Qwen · ggml-org
Params1.73B
Context32K
Best fitBF16 4.9 GB
LL
Hermes 3 Llama 3.1 8B
Llama · NousResearch
Params8B
Context128K
Best fitQ5_K_M 7.3 GB