GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
NE
nexus legal
Nexus · King3Djbl
Params1.54B
Context32K
Best fitF16 3.8 GB
JI
JiRackUltra 7b
Jirackultra · CMSManhattan
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
QW
Qwen2.5 Coder 7B Instruct Uncensored
Qwen · Fbjfhvgv
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
QW
Qwen3 0.6B Base
Qwen · mradermacher
Params596M
Context32K
Best fitF16 2.6 GB
GE
Gemma 3 270m It Qat Q4_0 Unquantized
Gemma · ggml-org
Params268M
Context32K
Best fitQ4_0 0.9 GB
KE
Kev-0.8B
Kev · ggml-org
Params753M
Context256K
Best fitBF16 2.4 GB
GE
Gemma-3N-E2B-It
Gemma · unsloth
Params4.46B
Context32K
Best fitQ4_0 4.0 GB
QW
Qwen3.8 27B Heretic Ara
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
QW
Qwen2.5-VL-7B-Instruct
Qwen · Mungert
Params7.6B
Context125K
Best fitIQ2_S 4.0 GB
QW
Qwen3.8 27B Heretic Abliterated Uncensored
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
TI
TildeOpen 30B Instruct LV
Tildeopen · mradermacher
Params30.7B
Context64K
Q4_K_M18.5 GB
QW
Qwen3.5 4B MultiDistill 3xMerge UNCENSORED Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
FR
FrogNano 4B Heretic
Frognano · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
QW
Qwen3.5 2B Base Heretic
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
GR
Granite 3.1 1b A400M Instruct
Granite · bartowski
Params1.33B
Context128K
Best fitF16 3.6 GB
QW
Qwen3.8 Flash Next Rulith EXL3
Qwen · Rulith
Params177B
Context256K
Smallest2.6 GB
YI
01-ai_yi-6b-200k
Yi · TheBloke
Params6.06B
Context195K
Best fitQ3_K_M 4.0 GB
TE
Ternary Bonsai 2 27B DFlash2
Ternary · ProCreations
Params1.92B
Context256K
Best fitQ8_0 2.7 GB
QW
Greko 1.7b Beta Merged
Qwen · geolazagr
Params1.72B
Context40K
Best fitIQ4_NL 2.5 GB
GR
granite-8b-code-instruct
Granite · mradermacher
Params8.05B
Context4K
Best fitIQ2_XXS 3.9 GB
GE
Gemmasutra Small 4B v1
Gemma · TheDrummer
Params3.86B
Context8K
Best fitQ3_K_M 4.0 GB
GE
Gemma 4 E4B it Artemis
Gemma · turintech
Params7.46B
Context128K
Best fitQ4_0 0.7 GB
QW
Qwen2.5 VL 7B Instruct
Qwen · DhruvalLabs
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
YI
Yi-1.5-9B-Chat
Yi · bartowski
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
QW
Qwen3-8B-128K
Qwen · unsloth
Params8.19B
Context128K
Best fitIQ1_S 4.0 GB
OR
Ornith1.0 9B 3MPER0RR Abliterated
Ornith · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen3 VL Reranker 2B
Qwen · mradermacher
Params1.72B
Context256K
Best fitQ6_K 2.9 GB
MI
Mistral-Large-4-1T-A52B-DSpark
Mistral · unsloth
Params1050B
Context1024K
Best fitQ8_0 2.8 GB
QW
Qwen3.5 122B A10B Heretic
Qwen · mradermacher
Params122B
Context256K
Q4_K_M74.2 GB
QW
Sperma Aj Qwen3.8 27b
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB