GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
ST
Step 3.5 Flash REAP 149B A11B i1
Step · mradermacher
Params149B
Context256K
Q4_K_M89.9 GB
GGUF
AU
Aura 4B
Aura · bartowski
Params4.51B
Context128K
Best fitQ3_K_L 4.0 GB
RoleplayTool use
QW
dgpl linux assistant 0.8b
Qwen · dgpl
Params752M
Context256K
Best fitF16 2.4 GB
ReasoningTool useVision
ER
Erk 14B
Erk · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
GGUF
LL
Llama-3.1-Argunaut-1-8B-HIRPO
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
Reasoning
ST
L3.1 Storniitova 8B i1
Storniitova · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
LL
dolphin 2.9.4 llama3.1 8b i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
Coding
QW
Qwen3 VL 8B NSFW Caption V4.5
Qwen · mradermacher
Params8.19B
Context256K
Best fitIQ1_M 4.0 GB
Vision
GP
Gpt Oss 20b RichardErkhov Heresy
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
GGUF
PY
Pythia 160m
Pythia · mradermacher
Params162M
Context2K
Best fitF16 1.1 GB
GGUF
LO
LongWriter V 7B
Longwriter · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
GGUF
QW
Qwen2-7B-Instruct
Qwen · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
PE
Pelican1.0 VL 235B A22B FC
Pelican · mradermacher
Params235B
Context256K
Q4_K_M142.2 GB
Vision
MI
MissLizzy_7b_HF
Misslizzy · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
QW
Qwen3 1.7B Jailbreak Reasoning
Qwen · mradermacher
Params1.72B
Context40K
Best fitQ8_0 3.3 GB
ReasoningTool use
QW
I2S Qwen3 8B Confidence
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
ReasoningTool use
QW
Qwen3.5 9B heretic imatrix
Qwen · FadedRedStar
Params456M
Context256K
Best fitQ8_0 3.7 GB
CodingVision
CU
CursorCore QW2.5 7B i1
Cursorcore · mradermacher
Params7.61B
Context32K
Best fitQ2_K 4.0 GB
Coding
QW
Qwen3.5 9B Polaris HighIQ THINKING
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningRoleplay
QW
Qwen2.5 1.5B Instruct Abliterated
Qwen · mradermacher
Params1.54B
Context32K
Best fitQ6_K 2.0 GB
GGUF
DA
Darwin 35B A3B Opus
Darwin · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
ReasoningTool useVision
QW
Bharat Tiny LLM v3
Qwen · eulogik
Params1.72B
Context32K
Best fitQ4_K_M 2.5 GB
Tool use
SP
Spark-X2.6-4B
Spark · Mungert
Params4.11B
Context1024K
Best fitQ3_K_M 4.0 GB
ReasoningTool use
QW
Qwen3.5 9B heretic
Qwen · FadedRedStar
Params8.95B
Context256K
Best fitQ8_0 3.7 GB
CodingVision
QW
Qwen3 32B LoRA ECHO KK GRPO
Qwen · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
GGUF
YI
Yi-1.5-9B-Chat
Yi · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
GGUF
TI
Tinyllama 15M
Tinyllama · QuantFactory
Params15M
Context256
Best fitQ2_K 0.6 GB
GGUF
OM
OmniCoder Queen 9B
Omnicoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingRoleplayVision
QW
UwU Qwen3.5 27B v0.1
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
ReasoningRoleplay
ZE
Zen4 Coder
Zen · mradermacher
Params79.7B
Context256K
Q4_K_M48.5 GB
Coding

Other memory sizes

Keep browsing