GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
CH
Chiboard 1.1 0.8b
Chiboard · mradermacher
Params752M
Context256K
Best fitQ4_1 1.5 GB
QW
alist food text qwen35
Qwen · NateRunsA-LIST
Params44M
Context256K
Best fitQ4_K_M 2.2 GB
HE
HERO 4B
Hero · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
LL
Llama 3.2 1B Instruct
Llama · gaianet
Params1.24B
Context128K
Best fitF16 3.2 GB
NA
NaNovel 35B A3B
Nanovel · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen3-4B-Nymphaea-RP
Qwen · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
LF
LFM2.5 2.6B UNCENSORED ABLITERATED PHILADELPHIA CLASS
Lfm · mradermacher
Params2.7B
Context125K
Best fitQ3_K_L 4.0 GB
VI
VieNeu TTS v2 Turbo
Vieneu · pnnbao-ump
Params112M
Context4K
Smallest0.2 GB
MA
MagpieLM 8B Chat v0.1 i1
Magpielm · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
NE
Nemotron 3.5 Embed 1B BF16
Nemotron · W4D
Params1.14B
Context256K
Best fitBF16 3.3 GB
CA
CantoneseLLM v2.0 30B A3B Thinking Chat Vector Merged
Cantonesellm · mradermacher
Params30.5B
Context40K
Q4_K_M18.6 GB
LL
Llama-3-8B-Instruct-MopeyMule
Llama · legraphista
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
MI
midashenglm 0.6b
Midashenglm · mispeech
Params596M
Context40K
Best fitBF16 2.6 GB
QW
Qwen3 VL 4B Instruct Abliterated v1
Qwen · prithivMLmods
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
QW
Qwable v1
Qwable · bartowski
Params35.5B
Context256K
Q4_K_M21.9 GB
MO
Morpho 1.5B Distill
Morpho · mradermacher
Params1.54B
Context128K
Best fitQ6_K 2.0 GB
OV
OvisOCR2
Ovisocr · bartowski
Params752M
Context256K
Best fitBF16 2.4 GB
LL
Meta-Llama-3-8B-Instruct-abliterated-v3
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
DE
DeepSeek R1 Distill Qwen 1.5B uncensored
DeepSeek · mradermacher
Params1.78B
Context128K
Best fitQ8_0 2.6 GB
QW
Qwen3 1.7b Abliterated
Qwen · mradermacher
Params1.72B
Context40K
Best fitQ6_K 2.9 GB
LF
LFM2.5 24B A2B Uncensored
Lfm · mradermacher
Params24B
Context128K
Q4_K_M14.5 GB
HY
Hy-Mt2-1.8B
unsloth
Params1.79B
Context256K
Best fitQ8_K_XL 3.4 GB
QW
Qwen3-Vl-8B-Instruct-1M
Qwen · unsloth
Params8.19B
Context977K
Best fitIQ1_S 4.0 GB
LO
Loopl 0.8b
Loopl · mradermacher
Params752M
Context256K
Best fitQ5_K_S 1.5 GB
LF
LFM2 VL 3B
Lfm · LiquidAI
Params2.57B
Context125K
Best fitQ4_0 4.0 GB
MI
RenCoder Ministral 8B Instruct 2410
Mistral · mradermacher
Params8B
Context32K
Best fitIQ1_M 3.9 GB
ME
Mephisto 1.0 4B
Mephisto · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
ME
Mephisto 1.5 4B
Mephisto · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
MI
ANITA NEXT 24B Magistral 2506 VISION ITA
Mistral · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
IN
Internlm2 Math 7b
InternLM · mradermacher
Params7.74B
Context8K
Best fitIQ2_XS 4.0 GB