GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
MI
MiniCPM V 4.7 35B A3B
MiniCPM · openbmb
Params34.7B
Context256K
Best fitF16 2.0 GB
GR
Granite 4.2 3b
Granite · ibm-granite
Params3.66B
Context128K
Best fitQ5_1 3.9 GB
LF
LFM2.5 Audio 1.5B
Lfm · LiquidAI
Params190M
Context125K
Best fitF16 3.0 GB
QW
Qwen3.5-0.8B
Qwen · unsloth
Params752M
Context256K
Best fitBF16 2.4 GB
QW
Ternary-Bonsai-2-2B
Qwen · prism-ml
Params1.88B
Context256K
Smallest0.4 GB
GP
gpt-oss-20b
gpt-oss · ggml-org
Params870M
Context128K
Best fitBF16 2.2 GB
GE
Gemma 3 1b It
Gemma · ggml-org
Params1B
Context32K
Best fitF16 2.7 GB
SM
TwIL LM3
SmolLM · webAI-Official
Params3.08B
Context64K
Best fitQ6_K 3.6 GB
QW
Qwen2.5-Vl-7B-Instruct
Qwen · unsloth
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
GP
gpt-oss-120b
gpt-oss · ggml-org
Params800M
Context128K
Best fitBF16 2.1 GB
QW
qwen2.5-3b-instruct
Qwen · Qwen
Params3.4B
Context32K
Best fitQ6_K 3.6 GB
QW
Qwen2.5 Coder 1.5B Instruct GGUF
Qwen · Qwen
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
QW
Qwen2.5 7B Instruct
Qwen · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
MI
s1 mini
Mini · superwhisper
Params752M
Context40K
Best fitF16 2.9 GB
SP
Spark X2.5 1.7B
Spark · XHToken
Params1.71B
Context1024K
Best fitQ8_0 2.8 GB
QW
Qwen3-1.7B
Qwen · unsloth
Params1.72B
Context40K
Best fitQ8_K_XL 3.8 GB
QW
Qwen3 0.6B Instruct
Qwen · Qwen
Params596M
Context40K
Best fitQ8_0 2.1 GB
GE
gemma 4 12B it qat
Gemma · unsloth
Params440M
Context256K
Best fitBF16 2.4 GB
MI
MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking
MiniCPM · GnLOLot
Params1.08B
Context128K
Best fitF16 2.9 GB
BG
bge small en v1.5
BGE · unsloth
Params33M
Context512
Best fitF16 0.7 GB
MU
Muse Glimmer 30B
Muse · meta-models
Params27.9B
Context128K
Best fitQ4_K_M 2.3 GB
QW
Qwen3-Vl-8B-Instruct
Qwen · unsloth
Params8.19B
Context256K
Best fitIQ1_S 4.0 GB
MX
mxbai-embed-large-v1
Mxbai · ChristianAzinn
Params334M
Context512
Q4_K_M0.2 GB
CA
Carbon 3B
Carbon · HuggingFaceBio
Params3.45B
Context32K
Best fitQ6_K 3.8 GB
MI
MiniCPM5 1B Claude Opus Fable5 Thinking
MiniCPM · GnLOLot
Params1.08B
Context128K
Best fitF16 2.9 GB
EX
Experiment26 7B passthrough 10slice
Experiment · babybirdprd
Params7B
Context—
Best fitQ4_K_M 0.5 GB
QW
Qwen3.5 9B
Qwen · bartowski
Params9.2B
Context256K
Q4_K_M6.2 GB
LF
LFM2.5 24B A2B Uncensored
Lfm · AtomicChat
Params24B
Context128K
Q4_K_M15.4 GB
QW
Qwen3Vl 2b Instruct
Qwen · Qwen
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
QW
Qwen3.8 27B Uncensored
Qwen · azukivc
Params27.3B
Context256K
Best fitF16 3.6 GB