GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Strand-Rust-Coder-14B-v1
Qwen · Mungert
Params14.8B
Context32K
Q4_K_M9.3 GB
Coding
QW
Fathom-Search-4B
Qwen · Mungert
Params4B
Context40K
Best fitIQ4_XS 4.0 GB
GGUF
HU
Huihui3.5 67B A3B Abliterated
Huihui · mradermacher
Params67B
Context256K
Q4_K_M40.8 GB
GGUF
HO
Sakura K2 Horizon 3.7B
Horizon · webmp3
Params5.06B
Context512K
Smallest2.2 GB
GGUF
MI
Ministral 3 14B Reasoning 2512 Absolute Heresy
Mistral · mradermacher
Params13.5B
Context256K
Q4_K_M8.2 GB
Reasoning
QW
Qwen2.5 Coder 0.5B
Qwen · bartowski
Params494M
Context32K
Best fitF16 1.6 GB
Coding
QW
Qwen3 8B Abliterated
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
GGUF
SP
Spark X2.6 4B Uncensored
Spark · mradermacher
Params4.11B
Context1024K
Best fitQ3_K_M 3.9 GB
ReasoningTool use
QW
Qwen3 0.6B Heretic Abliterated Uncensored
Qwen · Andycurrent
Params596M
Context40K
Q4_K_M0.4 GB
GGUF
QW
rombos Replete Coder Qwen2 1.5b i1
Qwen · mradermacher
Params1.54B
Context128K
Best fitQ6_K 2.0 GB
CodingTool use
YI
Mahou-1.3-yi-9B
Yi · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
GGUF
AD
AdaGuard 8B
Adaguard · mradermacher
Params8.19B
Context32K
Best fitIQ1_M 4.0 GB
GGUF
FA
Falcon3 1B Instruct
Falcon · bartowski
Params1.67B
Context8K
Best fitQ8_0 2.9 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
AR
Arcee-Agent
Arcee · crusoeai
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
GGUF
AY
aya-23-8B
Aya · legraphista
Params8B
Context8K
Best fitIQ1_M 3.9 GB
GGUF
AR
ArrowPro-7B-KillerWhale
Arrowpro · mmnga
Params7.24B
Context4K
Best fitIQ2_S 3.9 GB
GGUF
QW
Qwen3 32B VL GLM 4.7 Flash HI16 Heretic Uncensored Thinking
Qwen · mradermacher
Params32.8B
Context256K
Q4_K_M19.8 GB
ReasoningRoleplayVision
QW
Qwen3.5 27B Abliterated Ko
Qwen · mradermacher
Params26.9B
Context256K
Q4_K_M16.5 GB
ReasoningTool use
QW
Qwen3-8B
Qwen · ggml-org
Params8.19B
Context40K
Best fitBF16 2.9 GB
GGUF
DA
Darwin 4B Chimera
Darwin · bartowski
Params4B
Context40K
Best fitQ3_K_L 4.0 GB
Reasoning
TI
py007_tinyllama-1.1b-intermediate-step-715k-1.5t
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
Coding
LL
DarkIdol Llama 3.1 8B Instruct 1.0 Uncensored i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
Roleplay
MI
Ministral 3 14B Instruct 2512 Absolute Heresy
Mistral · mradermacher
Params13.5B
Context256K
Q4_K_M8.2 GB
GGUF
NO
Novaeus Promptist 7B Instruct i1
Novaeus · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
NE
neutts nano q4
Neutts · neuphonic
Params229M
Context—
Best fitQ4_0 0.7 GB
GGUF
IR
Irene-RP-v2-7B_128k
Irene · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
Roleplay
LL
Llama 3.2 1B Instruct
Llama · legraphista
Params1.24B
Context128K
Best fitBF16 3.2 GB
GGUF
QW
Qwen2.5 7B
Qwen · bartowski
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
Roleplay
DW
Dwight 4b
Dwight · bartowski
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
GGUF

Other memory sizes

Keep browsing