GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
EV
EAN 7B i1
Evan · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
SU
SummLlama3 8B i1
Summllama · mradermacher
Params8.03B
Context8K
Best fitIQ2_XXS 4.0 GB
QW
Trohrbaugh Qwen3.5 122B A10B Heretic
Qwen · mradermacher
Params122B
Context256K
Q4_K_M74.2 GB
QW
Qwen3-4B-128K
Qwen · unsloth
Params4B
Context128K
Best fitIQ4_XS 4.0 GB
TI
Tiamat-7b-1.1-DPO
Tiamat · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
LL
Llama-3-Patronus-Lynx-8B-Instruct
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
RO
ROMA 7B
Roma · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GR
GRM 2.6 Plus 0628
Grm · bartowski
Params27.3B
Context256K
Q4_K_M17.8 GB
SM
SmolVLM Instruct
Smolvlm · ggml-org
Params1.81B
Context16K
Best fitQ8_0 4.0 GB
KA
KAI 7B v0.1
Kai · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
LF
LFM2 1.2B Tool
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
MO
Mox Small 1
Mox · mradermacher
Params32.2B
Context64K
Q4_K_M19.5 GB
MI
MiniMax M2.1
Minimax · mradermacher
Params230B
Context192K
Q4_K_M138.3 GB
HU
Huihui Agents A1 Abliterated
Huihui · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
OP
OpenCoder 1.5B Instruct
Opencoder · lmstudio-community
Params1.91B
Context4K
Best fitQ4_K_M 3.7 GB
MI
Devstral 2 123B Instruct 2512
Mistral · bartowski
Params125B
Context256K
Q4_K_M74.9 GB
VI
Vietnamese 4B Chat Abliterated
Vietnamese · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
CA
calme-2.3-legalkit-8b
Calme · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
QW
Liberation Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
HY
HY MT1.5 1.8B
mradermacher
Params1.79B
Context256K
Best fitQ8_0 2.9 GB
PR
Prashna 7b
Prashna · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
MA
Mario Html v2 9B
Mario · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
KA
KAT Ornith HERO 35B A3B Fusion
Kat · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
DA
DataPilot ArrowPro 7B RobinHood
Datapilot · mmnga
Params7.24B
Context4K
Best fitIQ2_S 3.9 GB
MO
Mox Tiny 1
Mox · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
QW
Qwen3.5 0.8B Finetuned Lora Uncensored
Qwen · mradermacher
Params752M
Context256K
Best fitQ5_K_S 1.5 GB
QW
Qwen2.5 0.5B Instruct heretic
Qwen · saidutta69
Params494M
Context32K
Q4_K_M0.4 GB
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · mradermacher
Params8.19B
Context256K
Best fitIQ1_M 4.0 GB
QW
Qwen3 VL 4B Instruct Abliterated
Qwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
QW
flux2 klein 4B uncensored text encoder
Qwen · Cordux
Params4B
Context—
Best fitQ4_0 3.2 GB