GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
DA
danube2-1.8b-Tess-v1.5
Danube · mradermacher
Params1.83B
Context8K
Best fitQ8_0 3.0 GB
QW
Qwen2.5 Coder 1.5B Unsensored DPO
Qwen · mradermacher
Params1.54B
Context32K
Best fitQ6_K 2.0 GB
GE
gemma 2 2b rebus solver fp16
Gemma · mradermacher
Params2.61B
Context8K
Best fitQ6_K 3.5 GB
YI
Yi-9B-Coder
Yi · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
QW
Qwen3 4B Instruct 2507 Abliterated
Qwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
GE
gemma 4 E4B it qat q4 0 heretic
Gemma · FadedRedStar
Params7.46B
Context128K
Best fitQ4_0 0.8 GB
MI
Mizan v3 3B
Mizan · bartowski
Params2.9B
Context256K
Best fitQ6_K_L 3.9 GB
GP
Gpt Oss 20b Uncensored Heretic
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
QW
Josiefied Qwen3 8B Abliterated v1
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
CO
Yi Coder 1.5B Chat
Coder · mradermacher
Params1.48B
Context128K
Best fitQ8_0 3.7 GB
SM
SmolLM2 1.7B Instruct Uncensored
SmolLM · mradermacher
Params1.81B
Context8K
Best fitQ8_0 4.0 GB
QW
Huihui Qwen3 1.7B Abliterated v2
Qwen · mradermacher
Params1.72B
Context40K
Best fitQ8_0 3.3 GB
IN
L3.1 Instruct Guru 8B
Instruct · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
QW
Qwen3.5 35B A3B Claude 4.6 Opus Reasoning Distilled
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Ektome Qwen3 30B A3B PristinelyUncensored
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
QW
Qwen3.5 24B A3B Claude Opus Gemini 3.1 Pro Reasoning Distilled Heretic
Qwen · mradermacher
Params24.5B
Context256K
Q4_K_M15.0 GB
QW
QwQ LCoT2 7B Instruct i1
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
LF
LFM2.5-350M
Lfm · ensu-app
Params354M
Context125K
Best fitBF16 1.7 GB
AR
Armenian GPT 1.0 3B
Armenian · ArmGPT
Params3.43B
Context256K
Best fitQ4_K_M 3.5 GB
OM
OmniLing V1 8b experimental i1
Omniling · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GR
granite 4.2 3b
Granite · NANI-Nithin
Params3.66B
Context128K
Best fitQ5_1 3.9 GB
QW
Qwen2.5 0.5B
Qwen · mradermacher
Params494M
Context32K
Best fitBF16 1.6 GB
QW
Qwen3 Yoyo V3 42B A3B Thinking Total Recall
Qwen · mradermacher
Params42.4B
Context986K
Q4_K_M25.7 GB
MI
Mistral Small 3.2 24B Instruct 2506 Ultra Uncensored Heretic
Mistral · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
EU
Eurus 2 7B PRIME
Eurus · bartowski
Params7.62B
Context4K
Best fitQ2_K 4.0 GB
LL
Llama 3.2 3B Instruct heretic
Llama · saidutta69
Params3.21B
Context128K
Best fitQ5_K_M 3.8 GB
MI
Mistral-7B-v0.3
Mistral · legraphista
Params7.25B
Context32K
Best fitIQ2_S 3.9 GB
MI
Mistral 7B Instruct KhanAcademy v0.2 i1
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
QW
Qwen3.5 9B Claude 4.6 HighIQ INSTRUCT HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ1_S 4.0 GB
QW
Qwen2.5 0.5B Instruct
Qwen · second-state
Params494M
Context32K
Best fitF16 1.6 GB