GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
QW
Qwen3.6 35B A3B DFlash
Qwen · Anbeeld
Params386M
Context256K
Best fitBF16 1.5 GB
LL
Llama3-8B-Instruct-Replete-Adapted
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
CH
Chaotic Harmony 24B
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
LL
shisa v2.1 llama3.2 3b
Llama · mradermacher
Params3.21B
Context128K
Best fitQ5_K_M 3.8 GB
TE
TerminalHorizon 27B
Terminalhorizon · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
QW
qwen38 flash next v3
Qwen · nitinpanj
Params177B
Context256K
Best fitQ4_K_M 3.2 GB
ZR
ZR1 1.5B
bartowski
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
QW
Qwen3.5 4B Python Coder
Qwen · Abiray
Params4.2B
Context256K
Best fitQ3_K_M 3.8 GB
GR
Grug 27b v1.1
Grug · bartowski
Params27B
Context256K
Q4_K_M17.5 GB
QW
ISO27001 Qwen2.5 0.5B Edge
Qwen · sallani
Params494M
Context32K
Best fitQ4_K_M 1.0 GB
GP
Gpt Oss 20b Derestricted
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
LL
Llama 3.3 8B Instruct 128K Jbliterated
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
SE
SEX ROLEPLAY 3.2 1B i1
Sex · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
LF
LFM2.5 1.2B JP
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
NE
Next 1b
Next · mattritchey
Params1B
Context32K
Best fitQ4_K_M 1.5 GB
QW
X Coder RL Qwen3 8B
Qwen · mradermacher
Params8.19B
Context32K
Best fitIQ1_M 4.0 GB
GR
granite 4.2 3b heretic
Granite · saidutta69
Params3.66B
Context128K
Best fitQ5_K_M 3.8 GB
LL
Llama3.2 24B A3B II Dark Champion INSTRUCT Heretic Abliterated Uncensored
Llama · mradermacher
Params18B
Context128K
Q4_K_M11.1 GB
AT
Atom 80b
Atom · mradermacher
Params80B
Context256K
Q4_K_M48.4 GB
QW
Qwen3.8 35B A3B Distill
Qwen · ngquocvinh
Params35.5B
Context256K
Q4_K_M21.7 GB
DE
Deepseek Coder 1.3b Instruct
DeepSeek · mradermacher
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
MI
Ministral 3 3B Reasoning UNCENSORED Heretic
Mistral · mradermacher
Params3.43B
Context256K
Best fitQ5_K_S 3.8 GB
VO
VoxCPM-0.5B
Voxcpm · bluryar
Params728M
Context32K
Best fitF32 3.5 GB
OM
OmniDimen 2 20B Emotion
Omnidimen · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
IC
IceEspressoRPv2 7b i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
LL
VersatiLlama Llama 3.2 3B Instruct Abliterated
Llama · QuantFactory
Params3.21B
Context128K
Best fitQ5_1 3.9 GB
MI
snap1 2B
MiniCPM · logitlab
Params2.52B
Context128K
Best fitQ8_0 3.5 GB
PY
Pythia 1b
Pythia · mradermacher
Params1.01B
Context2K
Best fitF16 3.6 GB
QW
X Coder SFT Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
QW
Qwen2.5 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB