GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
TI
abdgrt_tinyllama-2-1b-miniguanaco
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
QW
Swift 1.5 Qwen3.8 27B Abliterated
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
QW
Qwen3 ASR 1.7B
Qwen · mradermacher
Params2.03B
Context64K
Best fitQ8_0 3.6 GB
QW
Qwen3.6 35B A3B StyleTune
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen3.8 27B Heretic JP Roleplay NSFW DanbooruTags
Qwen · EmilyOfOz
Params26.9B
Context256K
Q4_K_M16.5 GB
LF
Lfm2-350M
Lfm · unsloth
Params354M
Context32K
Best fitF16 1.7 GB
LL
Llama-3-Luminurse-v0.1-OAS-8B
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
ZE
Zen Voyager
Zen · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
TR
Trion-M-7b
Trion · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
QW
Qwen2.5 VL Abliterated Caption
Qwen · prithivMLmods
Params3.09B
Context125K
Best fitQ2_K 3.8 GB
GP
GPT4chan 8B
Gpt · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GR
Grug 35b
Grug · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
AA
aac board generator 770m ptbr
Aac · tardellirs
Params772M
Context32K
Best fitF16 2.3 GB
QW
Qwen3.6 35B A3B
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen2.5 7B Instruct 1M Thinking Claude Gemini GPT5.2 DISTILL
Qwen · mradermacher
Params7.62B
Context986K
Best fitQ2_K 4.0 GB
QW
Qwen3.8 27B RANA Abliterated
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
GR
Granite 3.3 2b Instruct
Granite · bartowski
Params2.53B
Context128K
Best fitQ8_0 3.9 GB
GE
Gemma 3 1b It Heretic Extreme Uncensored Abliterated
Gemma · mradermacher
Params1B
Context32K
Best fitF16 2.7 GB
RA
Radon 35B Ultra X RU
Radon · mradermacher
Params30.5B
Context64K
Q4_K_M18.6 GB
QW
Qwen3.8 27B Uncensored
Qwen · Guile
Params27.3B
Context256K
Q4_K_M17.8 GB
HE
HelpingAI-9B
Helpingai · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
MI
MiniCPM V 4.7 35B A3B APEX I NanoPlus
MiniCPM · IsValorum
Params34.7B
Context256K
Best fitQ6_K 0.9 GB
GP
Gpt Oss 20b Tainted Heresy
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
QW
Qwen3.8 Flash Next MTP
Qwen · dzannotti
Params3.88B
Context256K
Best fitQ4_K_M 3.9 GB
GP
openai community gpt2 GGUF smashed
Gpt · PrunaAI
Params124M
Context1K
Best fitQ2_K 0.9 GB
MA
magnum v2 4b
Magnum · bartowski
Params4.51B
Context128K
Best fitQ3_K_L 4.0 GB
JI
jina reranker v1 tiny en
Jina · gpustack
Params33M
Context8K
Best fitQ2_K 0.6 GB
HE
Helvete Nano
Helvete · Xetro
Params1.88B
Context1024K
Best fitQ4_K_M 2.2 GB
QW
gte-Qwen2-1.5B-instruct
Qwen · RichardErkhov
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
MI
dolphin-2.9.3-mistral-7B-32k
Mistral · mradermacher
Params7.25B
Context32K
Best fitIQ2_S 3.9 GB