GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
KU
Kunoichi DPO v2 7B
Kunoichi · brittlewis12
Params7.24B
Context8K
Best fitIQ2_S 3.9 GB
DE
Josiefied DeepSeek R1 0528 Qwen3 8B Abliterated v1
DeepSeek · mradermacher
Params8.19B
Context128K
Best fitIQ1_M 4.0 GB
QW
Qwen3 Reranker 4B
Qwen · mradermacher
Params4B
Context40K
Best fitIQ4_XS 4.0 GB
QW
Models Qwen Qwen2.5 3B Instruct
Qwen · MaziyarPanahi
Params3.09B
Context32K
Best fitQ6_K 3.3 GB
RU
MN 12B Runeweaver RP RU
Runeweaver · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
TR
Trinity Mini
Trinity · bartowski
Params26B
Context128K
Q4_K_M15.9 GB
MI
Mistralai Shieldstral 1.0 3B
Mistral · Abiray
Params3.43B
Context256K
Best fitQ5_K_S 3.8 GB
QW
Qwen3.8 27B MindMeld OneJev1.6
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
PU
L3.1 Purosani i1
Purosani · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
LL
ToolACE 2.5 Llama 3.1 8B Heretic
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GE
Gemma 2 2b
Gemma · legraphista
Params2.61B
Context8K
Best fitQ6_K 3.5 GB
PI
Piaget 8B
Piaget · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
QW
GrayLine Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
QW
Qwen3.5 9B Kimi K3 Distilled
Qwen · somenetworktrouble
Params9.2B
Context256K
Best fitQ3_K_S 1.6 GB
QW
Qwen3.5 4b Agentic Coder v4
Qwen · mradermacher
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
QW
Qwen3 Reranker 4B
Qwen · giladgd
Params4B
Context40K
Best fitQ3_K_L 3.9 GB
DR
DrakIdol Roleplayer 1.0
Drakidol · mradermacher
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
GE
gemma 3 4b instruct
Gemma · lm-kit
Params3.88B
Context128K
Best fitQ4_K_S 4.0 GB
LL
Llama Calix CLM 1.0
Llama · calix-ai-labs
Params8.03B
Context128K
Best fitIQ2_XXS 4.0 GB
MI
Magistral Small 2507
Mistral · bartowski
Params23.6B
Context40K
Q4_K_M14.3 GB
LL
Llama 3.2 3B Agent007 Coder
Llama · QuantFactory
Params3.21B
Context128K
Best fitQ5_1 3.9 GB
QW
jinaai/test-qwen25-coder-jina-code-embeddings-1.5b
Qwen · jinaai
Params1.54B
Context32K
Best fitBF16 3.8 GB
PO
Polaris VGA 27B Post1.0e
Polaris · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
LL
Kiel 2.1 Sol
Llama · kiel2
Params3.2B
Context128K
Best fitQ4_K_M 3.5 GB
QW
Dolphin3.0 Qwen2.5 3B
Qwen · QuantFactory
Params3.4B
Context32K
Best fitQ6_K 3.6 GB
QW
Qwen3-Vl-4B-Thinking-1M
Qwen · unsloth
Params4B
Context977K
Best fitIQ4_XS 4.0 GB
YI
Yi 1.5 6B Chat
Yi · QuantFactory
Params6.06B
Context4K
Best fitQ3_K_M 4.0 GB
LL
Hermes 2 Theta Llama 3 70B
Llama · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
BG
bge-small-en-v1.5
BGE · ChristianAzinn
Params33M
Context512
Q4_K_M0.0 GB
QW
Qwen3.5 40B RoughHouse Claude 4.6 Opus Polar Deckard Uncensored Heretic Thinking
Qwen · mradermacher
Params39.1B
Context256K
Q4_K_M23.9 GB