GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.6 35B A3B DFlash
Qwen · Anbeeld
Params386M
Context256K
Best fitBF16 1.5 GB
CodingEmbedding
LL
Llama3-8B-Instruct-Replete-Adapted
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
CodingTool use
CH
Chaotic Harmony 24B
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
LL
shisa v2.1 llama3.2 3b
Llama · mradermacher
Params3.21B
Context128K
Best fitQ5_K_M 3.8 GB
GGUF
TE
TerminalHorizon 27B
Terminalhorizon · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF
QW
qwen38 flash next v3
Qwen · nitinpanj
Params177B
Context256K
Best fitQ4_K_M 3.2 GB
GGUF
ZR
ZR1 1.5B
bartowski
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
Coding
QW
Qwen3.5 4B Python Coder
Qwen · Abiray
Params4.2B
Context256K
Best fitQ3_K_M 3.8 GB
Coding
GR
Grug 27b v1.1
Grug · bartowski
Params27B
Context256K
Q4_K_M17.5 GB
ReasoningTool useVision
QW
ISO27001 Qwen2.5 0.5B Edge
Qwen · sallani
Params494M
Context32K
Best fitQ4_K_M 1.0 GB
GGUF
GP
Gpt Oss 20b Derestricted
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
GGUF
LL
Llama 3.3 8B Instruct 128K Jbliterated
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
SE
SEX ROLEPLAY 3.2 1B i1
Sex · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
Roleplay
LF
LFM2.5 1.2B JP
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
GGUF
NE
Next 1b
Next · mattritchey
Params1B
Context32K
Best fitQ4_K_M 1.5 GB
Coding
QW
X Coder RL Qwen3 8B
Qwen · mradermacher
Params8.19B
Context32K
Best fitIQ1_M 4.0 GB
Coding
GR
granite 4.2 3b heretic
Granite · saidutta69
Params3.66B
Context128K
Best fitQ5_K_M 3.8 GB
ReasoningRoleplayTool use
LL
Llama3.2 24B A3B II Dark Champion INSTRUCT Heretic Abliterated Uncensored
Llama · mradermacher
Params18B
Context128K
Q4_K_M11.1 GB
Roleplay
AT
Atom 80b
Atom · mradermacher
Params80B
Context256K
Q4_K_M48.4 GB
Reasoning
QW
Qwen3.8 35B A3B Distill
Qwen · ngquocvinh
Params35.5B
Context256K
Q4_K_M21.7 GB
GGUF
DE
Deepseek Coder 1.3b Instruct
DeepSeek · mradermacher
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
Coding
MI
Ministral 3 3B Reasoning UNCENSORED Heretic
Mistral · mradermacher
Params3.43B
Context256K
Best fitQ5_K_S 3.8 GB
Reasoning
VO
VoxCPM-0.5B
Voxcpm · bluryar
Params728M
Context32K
Best fitF32 3.5 GB
GGUF
OM
OmniDimen 2 20B Emotion
Omnidimen · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
GGUF
IC
IceEspressoRPv2 7b i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
LL
VersatiLlama Llama 3.2 3B Instruct Abliterated
Llama · QuantFactory
Params3.21B
Context128K
Best fitQ5_1 3.9 GB
GGUF
MI
snap1 2B
MiniCPM · logitlab
Params2.52B
Context128K
Best fitQ8_0 3.5 GB
GGUF
PY
Pythia 1b
Pythia · mradermacher
Params1.01B
Context2K
Best fitF16 3.6 GB
GGUF
QW
X Coder SFT Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
Coding
QW
Qwen2.5 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF

Other memory sizes

Keep browsing