GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5 4B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params4.21B
Context256K
Best fitIQ4_XS 4.0 GB
ReasoningRoleplay
QI
QiMing CognitiveForge 14B
Qiming · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
GGUF
QW
PocketWeights Qwen2.5 14B Coder Creative
Qwen · mradermacher
Params14.8B
Context986K
Q4_K_M9.0 GB
Coding
QW
Qwen2 VL 2B Instruct Abliterated
Qwen · mradermacher
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
Vision
QW
deepsek qwen1.5 vpn
Qwen · tsss1
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
GGUF
FO
Foresight 32B
Foresight · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
GGUF
QW
Qwen3 4B Instruct 2507 Abliterated
Qwen · mradermacher
Params4.02B
Context256K
Best fitIQ4_XS 4.0 GB
Tool use
BE
Behemoth X 123B v2
Behemoth · bartowski
Params123B
Context128K
Q4_K_M73.2 GB
GGUF
QW
Qwen.Qwen2.5 VL 3B Instruct
Qwen · DevQuasar-6
Params669M
Context125K
Best fitF32 3.5 GB
Vision
LL
Llm Jp 3.1 1.8b
Llm · mmnga
Params1.87B
Context4K
Best fitQ6_K 3.6 GB
GGUF
AP
Apertus 70B Instruct 2509
Apertus · bartowski
Params70.6B
Context64K
Q4_K_M43.7 GB
GGUF
CO
Yi Coder 9B
Coder · mradermacher
Params8.83B
Context128K
Best fitIQ2_XS 4.0 GB
Coding
LL
Llama 3.2 3B Instruct Uncensored
Llama · PurpleAILAB
Params3.21B
Context128K
Best fitQ4_K_M 3.5 GB
GGUF
GE
Gemma 3 12b It Vl GLM 4.7 Flash Polaris Thinking Heretic Uncensored
Gemma · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
ReasoningVision
TI
TinyEnsemble-3x1.1B-TinyMoE
Tinyensemble · mradermacher
Params2.62B
Context4K
Best fitQ8_0 3.5 GB
Tool use
LL
llama-3-Nephilim-v2-8B
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
DE
DeepSeek R1 Distill Qwen 1.5B Fully Uncensored
DeepSeek · mradermacher
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
Reasoning
QW
Qwen3 0.6B Gensyn Swarm tall tame panther
Qwen · 0xgr3y
Params596M
Context40K
Best fitF16 2.6 GB
ReasoningTool use
GI
ru girlfriend roleplay 1.5b gguf q8
Girlfriend · aiplatforms
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
RoleplayTool use
HA
Hathor v4.6 i1
Hathor · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
LL
Llama-3-Oasis-v1-OAS-8B
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
MI
Magistral Small 2507 Rebased Vision
Mistral · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
ReasoningVision
CO
Cosmos Reason1 7B Heretic
Cosmos · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
LL
Llama 3 Youko 8b Instruct
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct Uncensored
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
Coding
RW
rwkv 6 finch 1b6
Rwkv · latestissue
Params1.6B
Context1024K
Best fitQ8_0 2.5 GB
GGUF
QW
Qwen3.5 9B Cyberpunk2077 Xiaotian All4 5pass
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
OP
Openhands Lm 7b v0.1
Openhands · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
BA
BabyHercules 4x150M
Babyhercules · mradermacher
Params411M
Context2K
Best fitF16 1.5 GB
GGUF

Other memory sizes

Keep browsing