GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
LL
Impish_LLAMA_4B_V2
Llama · SicariusSicariiStuff
Params4.5B
Context128K
Best fitQ3_K_L 4.0 GB
QW
Models Qwen Qwen3 4B Thinking 2507
Qwen · MaziyarPanahi
Params4B
Context256K
Best fitQ3_K_L 3.9 GB
NE
NeoHorse-1-4B
Neohorse · ngquocvinh
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
TI
Tinylaya For Testing
Tinylaya · ggml-org
Params90M
Context8K
Best fitQ8_0 0.7 GB
QW
Qwen3.8 27B Pi
Qwen · bartowski
Params27.3B
Context256K
Q4_K_M17.4 GB
SM
SmolVLM2 500M Video Instruct
Smolvlm · ggml-org
Params409M
Context8K
Best fitF16 1.7 GB
SM
SmolVLM 256M Instruct
Smolvlm · ggml-org
Params163M
Context8K
Best fitF16 1.0 GB
LF
Lfm2.5-Vl-3B
Lfm · unsloth
Params2.7B
Context125K
Best fitIQ4_XS 4.0 GB
OR
Ornith 1.5 122B UNCENSORED
Ornith · dealignai
Params125B
Context256K
Best fitQ8_0 2.3 GB
QW
Qwen2.5 Math 7B Instruct
Qwen · bartowski
Params7.62B
Context4K
Best fitQ2_K 4.0 GB
MI
Ministral 3 3B Instruct 2512
Mistral · bartowski
Params3.43B
Context256K
Best fitQ5_K_L 3.9 GB
QW
Qwen3.5 122B A10B
Qwen · bartowski
Params125B
Context256K
Q4_K_M77.6 GB
LL
dolphin 2.9 llama3 8b
Llama · bartowski
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
QW
Qwen3 0.6B Abliterated
Qwen · mradermacher
Params596M
Context40K
Best fitF16 2.6 GB
QW
Qwen3 0.6B Heretic Abliterated Uncensored
Qwen · mradermacher
Params596M
Context40K
Best fitQ5_K_S 1.9 GB
MI
Ministral-3-8B-Instruct-2512
Mistral · unsloth
Params8.49B
Context256K
Best fitIQ1_S 3.9 GB
QW
Qwen3 0.6B Heretic
Qwen · mradermacher
Params596M
Context40K
Best fitF16 2.6 GB
VI
VieNeu TTS v2
Vieneu · pnnbao-ump
Params294M
Context4K
Smallest0.2 GB
SM
SmolLM2 360M Instruct
SmolLM · HuggingFaceTB
Params362M
Context8K
Best fitQ8_0 1.2 GB
LL
Hermes 3 Llama 3.2 3b Base Fft Chatml Rerun 8xh100 E4 2024 11 24 4k Steps
Llama · bartowski
Params3.21B
Context128K
Best fitQ5_K_L 3.9 GB
QW
Qwen3 Next 80B A3B Instruct
Qwen · bartowski
Params80B
Context256K
Q4_K_M48.7 GB
QW
Qwen3.5-2B
Qwen · unsloth
Params1.94B
Context256K
Best fitQ8_K_XL 3.8 GB
QW
Qwen_Qwen3.5 0.8B
Qwen · lmstudio-community
Params752M
Context256K
Best fitQ8_0 1.7 GB
QW
Z Image Qwen 3 4b text encoder
Qwen · worstplayer
Params4B
Context40K
Best fitIQ4_XS 4.0 GB
QW
Qwen2.5 VL 3B Instruct
Qwen · lmstudio-community
Params3.09B
Context125K
Best fitQ6_K 3.3 GB
LL
testmod koen llama2 13b dpotrain testver
Llama · RichardErkhov
Params13B
Context—
Best fitQ3_K_M 2.9 GB
OR
Ornith 1.5 9B
Ornith · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
QW
Qwen2.5 Coder 32b Instruct Heretic Sft
Qwen · mradermacher
Params32.8B
Context32K
Q4_K_M19.9 GB
QW
Huihui Qwen3 4B Instruct 2507 Abliterated
Qwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
GE
Huihui gemma 4 E4B it qat q4 0 unquantized abliterated
Gemma · huihui-ai
Params7.46B
Context128K
Best fitBF16 0.8 GB