GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
QW
Qwen2.5 VL 3B Instruct Abliterated
Qwen · huihui-ai
Params3.09B
Context125K
Best fitQ4_K_M 2.7 GB
QW
Qwen3 0.6B Heretic Heresy Edition
Qwen · mradermacher
Params596M
Context40K
Best fitF16 2.6 GB
XY
XYZ Aquila Mini
Xyz · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
LF
LFM3 1.7B
Lfm · LiquidAI
Params1.73B
Context256K
Best fitQ8_0 4.0 GB
AR
BAAI AREX 2
Arex · bartowski
Params27B
Context256K
Q4_K_M17.2 GB
BA
BarryFutureman NeuralTurdusVariant1 7B
Barryfutureman · RichardErkhov
Params7B
Context—
Best fitQ3_K_S 4.0 GB
QW
Qwen3.6 35B A3B Uncensored Heretic
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
MI
MiniCPM V 4_6
MiniCPM · openbmb
Params752M
Context256K
Best fitF16 2.4 GB
GE
Gemma-3-4B-It-Qat
Gemma · unsloth
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
QW
Qwen3.5-0.8B
Qwen · weareapexcreators
Params773M
Context256K
Best fitQ4_K_M 1.5 GB
QW
Qwen2 VL 2B Instruct
Qwen · bartowski
Params1.54B
Context32K
Best fitF16 3.8 GB
FL
Flint-40B-A2.5B-abliterated
Flint · dima6312
Params39.8B
Context256K
Q4_K_M24.2 GB
QW
Huihui Qwen3.5 35B A3B Abliterated
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Therapy 3.8
Qwen · Verdugie
Params26.9B
Context256K
Q4_K_M16.5 GB
LF
LFM2.5-350M
Lfm · weareapexcreators
Params354M
Context125K
Best fitQ8_0 1.4 GB
QW
Qwen3.8-27B-DFlash2
Qwen · perplexity-ai
Params27.3B
Context256K
Best fitQ8_0 3.8 GB
FL
RecursiveSI flint
Flint · bartowski
Params39.8B
Context256K
Q4_K_M25.3 GB
LF
LFM2 VL 1.6B
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
LL
VAGOsolutions Llama 3 SauerkrautLM 70b Instruct
Llama · RichardErkhov
Params6B
Context—
Best fitQ3_K_L 4.0 GB
PA
PaddleOCR VL 1.5
Paddleocr · PaddlePaddle
Params467M
Context128K
Smallest0.9 GB
GE
gemma-2b-it
Gemma · lmstudio-ai
Params2.51B
Context8K
Best fitQ8_0 3.3 GB
LF
LFM2.5 230M LocalAI
Lfm · weareapexcreators
Params230M
Context125K
Best fitQ8_0 1.2 GB
JI
jina reranker v1 tiny en
Jina · mradermacher
Params33M
Context8K
Best fitQ2_K 0.6 GB
MI
Mistral Small 4 119B 2603
Mistral · bartowski
Params119B
Context1024K
Q4_K_M72.6 GB
QW
Qwen3.5 27B Claude 4.6 Opus Reasoning Distilled
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
SP
Spark X2.6 4B
Spark · bartowski
Params4.11B
Context1024K
Best fitIQ4_XS 4.0 GB
QW
Qwen3.5-2B
Qwen · weareapexcreators
Params1.94B
Context256K
Best fitQ5_K_M 2.4 GB
LL
Llama 3.2 3B Instruct abliterated
Llama · QuantFactory
Params3.2B
Context128K
Best fitQ5_1 3.9 GB
QW
Qwen3.5 2B Claude 4.6 Opus Reasoning Distilled
Qwen · Jackrong
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
MI
MiniCPM5 2B LocalAI
MiniCPM · weareapexcreators
Params2.52B
Context128K
Best fitQ8_0 3.5 GB