GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3 1.7B
Qwen · ggml-org
Params2.03B
Context40K
Best fitF16 5.5 GB
QW
Qwen2.5 VL 3B Instruct
Qwen · ggml-org
Params3.09B
Context125K
Best fitF16 7.0 GB
VI
VieNeu TTS 0.3B q4
Vieneu · pnnbao-ump
Params294M
Context4K
Best fitQ4_0 0.8 GB
QW
Huihui Qwen3.8 Flash Next abliterated i1
Qwen · mradermacher
Params177B
Context—
Smallest0.6 GB
BG
bge-base-en-v1.5
BGE · CompendiumLabs
Params109M
Context512
Best fitF32 1.2 GB
QW
Qwen3.5 4B
Qwen · bartowski
Params4.33B
Context256K
Best fitQ8_0 6.2 GB
QW
Qwen Image 2.1
Qwen · leejet
Params7.12B
Context—
Best fitQ6_K 7.2 GB
GR
Granite 4.2 3B BE
Granite · taronaeo
Params3.66B
Context—
Best fitQ8_0 4.9 GB
IM
Z Image Turbo
Image · leejet
Params6.15B
Context—
Best fitQ8_0 7.9 GB
GE
Gemma 4 E2B It
Gemma · bartowski
Params4.65B
Context128K
Best fitQ8_0 6.1 GB
PH
Phi 4 Mini Instruct
Phi · unsloth
Params3.84B
Context128K
Best fitQ8_0 5.7 GB
QW
Qwen3 Reranker 0.6B
Qwen · ggml-org
Params596M
Context40K
Best fitQ8_0 2.1 GB
QW
Qwen Image 2.1 Text Encoder Heretic
Qwen · pottokao
Params8.2B
Context256K
Best fitQ4_K_M 6.7 GB
GE
Gemma-3-4B-It
Gemma · unsloth
Params3.88B
Context128K
Best fitQ8_K_XL 6.8 GB
TE
test model router download
Test · ggml-org
Params1M
Context2K
Best fitF16 0.5 GB
BO
Bonsai-8B
Bonsai · prism-ml
Params8.2B
Context64K
Best fitQ1_0 2.9 GB
QW
Qwen2.5 0.5B Instruct
Qwen · bartowski
Params494M
Context32K
Best fitF16 1.6 GB
MI
MiniCPM V 4.7 1B
MiniCPM · openbmb
Params1.01B
Context256K
Best fitF16 2.9 GB
QW
Qwen3 0.6B
Qwen · bartowski
Params752M
Context32K
Best fitBF16 2.9 GB
GE
Gemma 4 E2B
Gemma · lmstudio-community
Params4.65B
Context128K
Best fitQ8_0 6.1 GB
EM
embeddinggemma-2
Embeddinggemma · ggml-org
Params271M
Context256K
Best fitBF16 2.7 GB
QW
Qwen3.5 4B
Qwen · SoAIHQ
Params4.33B
Context256K
Best fitQ8_0 6.2 GB
QW
Qwen3.5-4B
Qwen · unsloth
Params4.33B
Context256K
Best fitQ8_K_XL 7.7 GB
SU
Supra2 100M Instruct
Supra · SupraLabs
Params101M
Context2K
Best fitF16 0.8 GB
DE
Deepseek-R1-Distill-Qwen-1.5B
DeepSeek · unsloth
Params1.78B
Context128K
Best fitBF16 4.3 GB
LF
LFM2.5 8B A1B
Lfm · LiquidAI
Params8.47B
Context125K
Best fitQ4_K_M 7.3 GB
PH
Phi 4 Mini Instruct
Phi · bartowski
Params3.84B
Context128K
Best fitQ8_0 5.7 GB
QW
nl2sh 1.5b Q4 K M
Qwen · ThorOdinson246
Params1.54B
Context32K
Best fitQ4_K_M 1.7 GB
GE
Gemma-3-12B-It-Qat
Gemma · unsloth
Params11.8B
Context128K
Best fitIQ2_XXS 7.3 GB
QW
Qwen3.5 0.8B Artemis
Qwen · turintech
Params752M
Context256K
Best fitQ5_K_M 1.5 GB