GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
QW
Qwen3.5 4B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params4.21B
Context256K
Best fitIQ4_XS 4.0 GB
QI
QiMing CognitiveForge 14B
Qiming · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
QW
PocketWeights Qwen2.5 14B Coder Creative
Qwen · mradermacher
Params14.8B
Context986K
Q4_K_M9.0 GB
QW
Qwen2 VL 2B Instruct Abliterated
Qwen · mradermacher
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
QW
deepsek qwen1.5 vpn
Qwen · tsss1
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
FO
Foresight 32B
Foresight · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
QW
Qwen3 4B Instruct 2507 Abliterated
Qwen · mradermacher
Params4.02B
Context256K
Best fitIQ4_XS 4.0 GB
BE
Behemoth X 123B v2
Behemoth · bartowski
Params123B
Context128K
Q4_K_M73.2 GB
QW
Qwen.Qwen2.5 VL 3B Instruct
Qwen · DevQuasar-6
Params669M
Context125K
Best fitF32 3.5 GB
LL
Llm Jp 3.1 1.8b
Llm · mmnga
Params1.87B
Context4K
Best fitQ6_K 3.6 GB
AP
Apertus 70B Instruct 2509
Apertus · bartowski
Params70.6B
Context64K
Q4_K_M43.7 GB
CO
Yi Coder 9B
Coder · mradermacher
Params8.83B
Context128K
Best fitIQ2_XS 4.0 GB
LL
Llama 3.2 3B Instruct Uncensored
Llama · PurpleAILAB
Params3.21B
Context128K
Best fitQ4_K_M 3.5 GB
GE
Gemma 3 12b It Vl GLM 4.7 Flash Polaris Thinking Heretic Uncensored
Gemma · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
TI
TinyEnsemble-3x1.1B-TinyMoE
Tinyensemble · mradermacher
Params2.62B
Context4K
Best fitQ8_0 3.5 GB
LL
llama-3-Nephilim-v2-8B
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
DE
DeepSeek R1 Distill Qwen 1.5B Fully Uncensored
DeepSeek · mradermacher
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
QW
Qwen3 0.6B Gensyn Swarm tall tame panther
Qwen · 0xgr3y
Params596M
Context40K
Best fitF16 2.6 GB
GI
ru girlfriend roleplay 1.5b gguf q8
Girlfriend · aiplatforms
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
HA
Hathor v4.6 i1
Hathor · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
LL
Llama-3-Oasis-v1-OAS-8B
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
MI
Magistral Small 2507 Rebased Vision
Mistral · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
CO
Cosmos Reason1 7B Heretic
Cosmos · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
LL
Meta Llama 3.1 8B Instruct
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
LL
Llama 3 Youko 8b Instruct
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
QW
Qwen2.5 Coder 7B Instruct Uncensored
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
RW
rwkv 6 finch 1b6
Rwkv · latestissue
Params1.6B
Context1024K
Best fitQ8_0 2.5 GB
QW
Qwen3.5 9B Cyberpunk2077 Xiaotian All4 5pass
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
OP
Openhands Lm 7b v0.1
Openhands · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
BA
BabyHercules 4x150M
Babyhercules · mradermacher
Params411M
Context2K
Best fitF16 1.5 GB