GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
DG
dgpl experimental 1
Dgpl · dgpl
Params550M
Context—
Best fitF16 1.8 GB
QW
Qwen_Qwen3.5 2B
Qwen · lmstudio-community
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
LI
H2O Lightning 4B
Lightning · h2oai
Params4.21B
Context256K
Best fitIQ3_XXS 3.8 GB
LL
Models Meta Llama Llama 3.2 3B Instruct
Llama · MaziyarPanahi
Params3.21B
Context128K
Best fitQ5_K_M 3.8 GB
PH
Phi 4 Mini Instruct
Phi · MaziyarPanahi
Params3.84B
Context128K
Best fitQ4_K_S 3.9 GB
GE
Winnow E4B
Gemma · EldanRing
Params78M
Context128K
Best fitBF16 0.8 GB
GL
GLM OCR
GLM · ggml-org
Params891M
Context128K
Best fitF16 2.9 GB
QW
Models Qwen Qwen2.5 7B Instruct
Qwen · MaziyarPanahi
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
LL
Llama-3.2-1B-Instruct
Llama · unsloth
Params1.24B
Context128K
Best fitF16 3.2 GB
DE
Deepseek-R1-Distill-Llama-8B
DeepSeek · unsloth
Params8B
Context128K
Best fitIQ1_M 3.9 GB
OR
Ornith 1.5 35B A3B
Ornith · bartowski
Params35.5B
Context256K
Q4_K_M21.9 GB
CO
Models 01 Ai Yi Coder 9B Chat
Coder · MaziyarPanahi
Params8.83B
Context128K
Best fitIQ2_XS 4.0 GB
QW
Models Qwen Qwen2.5 1.5B Instruct
Qwen · MaziyarPanahi
Params1.54B
Context32K
Q4_K_M1.0 GB
CO
Models 01 Ai Yi Coder 1.5B Chat
Coder · MaziyarPanahi
Params1.48B
Context128K
Best fitQ5_K_S 3.2 GB
QW
Qwen2.5 VL 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
MI
Models Mistralai Ministral 3 3B Reasoning 2512
Mistral · MaziyarPanahi
Params3.43B
Context256K
Best fitQ5_K_M 3.8 GB
GE
Models Google Gemma 2 2b It
Gemma · MaziyarPanahi
Params2.61B
Context8K
Best fitQ6_K 3.5 GB
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Best fitQ4_1 1.8 GB
GE
Models Google Gemma 3 1b It
Gemma · MaziyarPanahi
Params1B
Context32K
Q4_K_M0.8 GB
BE
BeaverAI NeMoist 21B v1a
Beaverai · RichardErkhov
Params21B
Context—
Best fitIQ3_M 0.9 GB
QW
Qwen3.8 27B Heretic JP Roleplay NSFW DanbooruTags
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
QW
Qwen3 TTS 12Hz 1.7B Base
Qwen · ggml-org
Params1.73B
Context32K
Best fitQ8_0 3.3 GB
YI
models--01-ai--Yi-1.5-6B-Chat
Yi · MaziyarPanahi
Params6.06B
Context4K
Best fitQ3_K_M 4.0 GB
MI
Mistral-7B-Instruct-v0.3
Mistral · thesven
Params7B
Context32K
Best fitBF16 1.6 GB
MI
models--mistralai--mathstral-7B-v0.1
Mistral · MaziyarPanahi
Params7.25B
Context32K
Best fitIQ2_XS 3.8 GB
LL
Llama 3.2 1B Instruct
Llama · hugging-quants
Params1.24B
Context128K
Best fitQ4_K_M 1.6 GB
OP
Tinyopenjev For Testing
Openjev · ggml-org
Params33M
Context256K
Best fitQ8_0 0.6 GB
QW
Qwen3.5 35B A3B
Qwen · bartowski
Params35.5B
Context256K
Q4_K_M22.3 GB
GL
Glm-4.6V-Flash
GLM · unsloth
Params9.4B
Context128K
Best fitIQ1_S 4.0 GB
QW
models--Qwen--Qwen2-7B-Instruct
Qwen · MaziyarPanahi
Params7.62B
Context32K
Best fitQ2_K 4.0 GB