GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
GE
gemma 4 E2B it qat MTP
Gemma · magnitudedev
Params77M
Context128K
Best fitQ4_0 0.6 GB
BU
Buddy E4B v2
Buddy · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
OE
Oellm 9b 128k Theta32M
Oellm · mradermacher
Params9.1B
Context128K
Q4_K_M5.7 GB
GP
Gpt2 Large
Gpt · mradermacher
Params838M
Context1K
Best fitQ6_K 2.7 GB
IR
Irbis-7b-v0.1
Irbis · mradermacher
Params7.47B
Context32K
Best fitIQ2_S 4.0 GB
GE
Gemma 4 E4B It Ultra Uncensored Heretic
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
GE
Gemma 4 E4B It The DECKARD Expresso Universe HERETIC UNCENSORED Thinking
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
QW
Qwen2.5 0.5B Instruct
Qwen · legraphista
Params494M
Context32K
Best fitBF16 1.6 GB
QW
Gliese Qwen3.5 9B Abliterated Caption
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ1_S 4.0 GB
JE
Jebadiah 9b v2 1
Jebadiah · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
QW
qwen rp 0.5b
Qwen · andybanana
Params494M
Context32K
Best fitQ4_K_M 1.0 GB
ST
Stellar Witch 12B Heretic
Stellar · mradermacher
Params12.2B
Context128K
Q4_K_M7.5 GB
GE
Gemma-4-E4B-It
Gemma · Hlebek
Params7.52B
Context128K
Best fitBF16 2.1 GB
CA
Carbon X
Carbon · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · second-state
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
GE
Gemini Nano Gemmafied
Gemma · mradermacher
Params3.24B
Context8K
Best fitQ6_K 3.4 GB
CO
Q2.5 ColdBrew R1 ForgeWide SAFE
Coldbrew · mradermacher
Params9.5B
Context128K
Best fitIQ2_XXS 3.9 GB
TR
Translategemma 4b It
Translategemma · mradermacher
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
CO
ContextPilot 14B
Contextpilot · bartowski
Params14.8B
Context40K
Q4_K_M9.3 GB
QW
Qwen3.5 27B Writer V2 Uncensored Heretic
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
LL
Llama 3.2 3B Instruct Abliterated
Llama · Hasaranga85
Params3.61B
Context128K
Best fitQ4_K_M 3.7 GB
GE
Locus Gemma 4 E2B
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
QW
Prettybird_Bce_Basic_Coder_8B
Qwen · mradermacher
Params7.6B
Context63K
Best fitQ2_K 4.0 GB
CY
Cydonia 24B v4.1
Cydonia · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
SM
SmolLM2-135M
SmolLM · ggml-org
Params135M
Context8K
Best fitBF16 1.0 GB
QW
Qwen3.5 2B
Qwen · gguf
Params1.94B
Context256K
Best fitQ8_0 3.0 GB
LL
Tiamat 8b 1.2 Llama 3 DPO i1
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
MI
Mistral Magistral Devstral Instruct FUSED CODER Reasoning 36B
Mistral · mradermacher
Params35.8B
Context128K
Q4_K_M21.7 GB
YI
Yi 1.5 9B Chat Abliterated
Yi · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
VE
Q25 1.5B VeoLu
Veolu · bartowski
Params1.78B
Context32K
Best fitQ8_0 2.6 GB