GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
MI
MiniCPM V 4.7 35B A3B CRACKv5 2p0
MiniCPM · dealignai
Params34.7B
Context256K
Best fitBF16 2.0 GB
DE
DeepSeek R1 Distill Qwen 1.5B
DeepSeek · bartowski
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
LF
LFM2.5 2.6B GGUF shards
Lfm · Arki05
Params2.7B
Context128K
Best fitIQ4_XS 4.0 GB
TE
HY MT1.5 1.8B
Tencent · tencent
Params1.79B
Context256K
Best fitQ8_0 2.9 GB
GE
gemma 2b fine tune
Gemma · arnavj007
Params2B
Context—
Best fitF16 0.5 GB
OR
OrcaCyber Nano 1.0
Orcacyber · orcarouter
Params1.92B
Context256K
Best fitQ8_0 2.8 GB
ME
Medgemma-4B-It
Medgemma · unsloth
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
TI
py007_tinyllama-1.1b-chat-v0.3
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
QW
Qwen2.5-VL-3B-Instruct
Qwen · Mungert
Params3.1B
Context125K
Best fitBF16 3.7 GB
EM
Embeddinggemma 300m Qat Q8_0 Unquantized
Embeddinggemma · ggml-org
Params303M
Context2K
Best fitQ8_0 1.0 GB
GE
Gemma 2 2b It
Gemma · bartowski
Params2.61B
Context8K
Best fitQ6_K_L 3.7 GB
QW
Qwen2.5 Coder 0.5B Instruct AWQ
Qwen · Qwen
Params630M
Context32K
Q4_K_M0.5 GB
QW
Qwen2.5 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GE
Huihui gemma 4 26B A4B it qat q4 0 unquantized abliterated
Gemma · huihui-ai
Params25.2B
Context128K
Best fitBF16 2.4 GB
TE
TerminalHorizon 122B A10B
Terminalhorizon · mradermacher
Params122B
Context256K
Q4_K_M74.2 GB
PR
Fish Speech S2 Pro Dual AR
Pro · rodrigomt
Params4.95B
Context32K
Best fitQ2_K 3.8 GB
LL
tiny llamas
Llama · ggml-org
Params24M
Context—
Smallest0.0 GB
QW
Qwen2.5 7B Instruct abliterated v2
Qwen · QuantFactory
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
LF
LFM2.5 VL 1.6B
Lfm · LiquidAI
Params1.17B
Context125K
Best fitBF16 3.9 GB
QW
Qwen3.5 Text 9B NSFW RP RolePlay
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Meta-Llama-3-8B-Instruct
Llama · bartowski
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
DE
DeepSeek V4.1 Flash 1017 abliterated
DeepSeek · orcarouter
Params748B
Context1024K
Smallest1.6 GB
NE
NVIDIA Nemotron 3.5 Super VL 120B A12B
Nemotron · unsloth
Params1.63B
Context1024K
Smallest1.3 GB
SM
SmolLM3 3B
SmolLM · ggml-org
Params3.08B
Context64K
Best fitQ4_K_M 3.0 GB
MX
mxbai embed xsmall v1
Mxbai · mixedbread-ai
Params24M
Context4K
Best fitF32 0.7 GB
QW
Qwen3.5 0.8B
Qwen · bartowski
Params773M
Context256K
Best fitBF16 2.5 GB
ST
stock report2
Stock · uiyong
Params—
Context—
Best fitQ8_0 0.5 GB
GE
Huihui gemma 4 31B it qat q4 0 unquantized abliterated
Gemma · huihui-ai
Params30.7B
Context128K
Best fitBF16 3.6 GB
KA
Karna 49M SFT Model
Karna · VarunHujan
Params72M
Context2K
Best fitQ8_0 0.7 GB
SA
Sage-A1.5
Sage · na0x2c6
Params35.5B
Context256K
Q4_K_M21.7 GB