GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
GE
gemma 4 E4B it qat
Gemma · unsloth
Params93M
Context128K
Best fitQ4_K_XL 4.8 GB
QW
Qwen3.6-35B-A3B
Qwen · ggml-org
Params1.9B
Context256K
Best fitBF16 4.9 GB
SN
snowflake arctic embed m v1.5
Snowflake · Snowflake
Params109M
Context512
Best fitF32 1.2 GB
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · bartowski
Params7.62B
Context128K
Best fitQ4_1 5.8 GB
BG
bge-large-en-v1.5
BGE · CompendiumLabs
Params334M
Context512
Best fitF32 2.6 GB
QW
Qwen3 1.7B
Qwen · ggml-org
Params2.03B
Context40K
Best fitF16 5.5 GB
QW
Qwen2.5 VL 3B Instruct
Qwen · ggml-org
Params3.09B
Context125K
Best fitQ8_0 4.1 GB
VI
VieNeu TTS 0.3B q4
Vieneu · pnnbao-ump
Params294M
Context4K
Best fitQ4_0 0.8 GB
QW
Huihui Qwen3.8 Flash Next abliterated i1
Qwen · mradermacher
Params177B
Context—
Smallest0.6 GB
BG
bge-base-en-v1.5
BGE · CompendiumLabs
Params109M
Context512
Best fitF32 1.2 GB
QW
Qwen3.5 4B
Qwen · bartowski
Params4.33B
Context256K
Best fitQ6_K_L 5.6 GB
QW
Qwen Image 2.1
Qwen · leejet
Params7.12B
Context—
Best fitQ4_0 5.2 GB
GR
Granite 4.2 3B BE
Granite · taronaeo
Params3.66B
Context—
Best fitQ8_0 4.9 GB
IM
Z Image Turbo
Image · leejet
Params6.15B
Context—
Best fitQ5_0 5.6 GB
GE
Gemma 4 E2B It
Gemma · bartowski
Params4.65B
Context128K
Best fitQ6_K_L 5.7 GB
PH
Phi 4 Mini Instruct
Phi · unsloth
Params3.84B
Context128K
Best fitQ8_0 5.7 GB
QW
Qwen3 Reranker 0.6B
Qwen · ggml-org
Params596M
Context40K
Best fitQ8_0 2.1 GB
GE
Gemma-3-4B-It
Gemma · unsloth
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
TE
test model router download
Test · ggml-org
Params1M
Context2K
Best fitF16 0.5 GB
BO
Bonsai-8B
Bonsai · prism-ml
Params8.2B
Context64K
Best fitQ1_0 2.9 GB
QW
Qwen2.5 0.5B Instruct
Qwen · bartowski
Params494M
Context32K
Best fitF16 1.6 GB
MI
MiniCPM V 4.7 1B
MiniCPM · openbmb
Params1.01B
Context256K
Best fitF16 2.9 GB
QW
Qwen3 0.6B
Qwen · bartowski
Params752M
Context32K
Best fitBF16 2.9 GB
GE
Gemma 4 E2B
Gemma · lmstudio-community
Params4.65B
Context128K
Best fitQ6_K 4.9 GB
EM
embeddinggemma-2
Embeddinggemma · ggml-org
Params271M
Context256K
Best fitBF16 2.7 GB
QW
Qwen3.5 4B
Qwen · SoAIHQ
Params4.33B
Context256K
Best fitQ4_K_M 4.4 GB
QW
Qwen3.5-4B
Qwen · unsloth
Params4.33B
Context256K
Best fitQ6_K_XL 5.9 GB
SU
Supra2 100M Instruct
Supra · SupraLabs
Params101M
Context2K
Best fitF16 0.8 GB
DE
Deepseek-R1-Distill-Qwen-1.5B
DeepSeek · unsloth
Params1.78B
Context128K
Best fitBF16 4.3 GB
PH
Phi 4 Mini Instruct
Phi · bartowski
Params3.84B
Context128K
Best fitQ8_0 5.7 GB