GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
TI
abdgrt_tinyllama-2-1b-miniguanaco
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
GGUF
QW
Swift 1.5 Qwen3.8 27B Abliterated
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
Reasoning
QW
Qwen3 ASR 1.7B
Qwen · mradermacher
Params2.03B
Context64K
Best fitQ8_0 3.6 GB
Vision
QW
Qwen3.6 35B A3B StyleTune
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
Roleplay
QW
Qwen3.8 27B Heretic JP Roleplay NSFW DanbooruTags
Qwen · EmilyOfOz
Params26.9B
Context256K
Q4_K_M16.5 GB
ReasoningRoleplayTool use
LF
Lfm2-350M
Lfm · unsloth
Params354M
Context32K
Best fitF16 1.7 GB
GGUF
LL
Llama-3-Luminurse-v0.1-OAS-8B
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
ZE
Zen Voyager
Zen · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
GGUF
TR
Trion-M-7b
Trion · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
QW
Qwen2.5 VL Abliterated Caption
Qwen · prithivMLmods
Params3.09B
Context125K
Best fitQ2_K 3.8 GB
Vision
GP
GPT4chan 8B
Gpt · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
GR
Grug 35b
Grug · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
CodingReasoningTool use
AA
aac board generator 770m ptbr
Aac · tardellirs
Params772M
Context32K
Best fitF16 2.3 GB
GGUF
QW
Qwen3.6 35B A3B
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
GGUF
QW
Qwen2.5 7B Instruct 1M Thinking Claude Gemini GPT5.2 DISTILL
Qwen · mradermacher
Params7.62B
Context986K
Best fitQ2_K 4.0 GB
ReasoningRoleplay
QW
Qwen3.8 27B RANA Abliterated
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
Vision
GR
Granite 3.3 2b Instruct
Granite · bartowski
Params2.53B
Context128K
Best fitQ8_0 3.9 GB
GGUF
GE
Gemma 3 1b It Heretic Extreme Uncensored Abliterated
Gemma · mradermacher
Params1B
Context32K
Best fitF16 2.7 GB
GGUF
RA
Radon 35B Ultra X RU
Radon · mradermacher
Params30.5B
Context64K
Q4_K_M18.6 GB
GGUF
QW
Qwen3.8 27B Uncensored
Qwen · Guile
Params27.3B
Context256K
Q4_K_M17.8 GB
ReasoningTool useVision
HE
HelpingAI-9B
Helpingai · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
GGUF
MI
MiniCPM V 4.7 35B A3B APEX I NanoPlus
MiniCPM · IsValorum
Params34.7B
Context256K
Best fitQ6_K 0.9 GB
Vision
GP
Gpt Oss 20b Tainted Heresy
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
GGUF
QW
Qwen3.8 Flash Next MTP
Qwen · dzannotti
Params3.88B
Context256K
Best fitQ4_K_M 3.9 GB
GGUF
GP
openai community gpt2 GGUF smashed
Gpt · PrunaAI
Params124M
Context1K
Best fitQ2_K 0.9 GB
GGUF
MA
magnum v2 4b
Magnum · bartowski
Params4.51B
Context128K
Best fitQ3_K_L 4.0 GB
GGUF
JI
jina reranker v1 tiny en
Jina · gpustack
Params33M
Context8K
Best fitQ2_K 0.6 GB
CodingEmbedding
HE
Helvete Nano
Helvete · Xetro
Params1.88B
Context1024K
Best fitQ4_K_M 2.2 GB
Roleplay
QW
gte-Qwen2-1.5B-instruct
Qwen · RichardErkhov
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
GGUF
MI
dolphin-2.9.3-mistral-7B-32k
Mistral · mradermacher
Params7.25B
Context32K
Best fitIQ2_S 3.9 GB
CodingTool use

Other memory sizes

Keep browsing