SmolLM GGUF models

1353 GGUF models in the SmolLM family, quantized by 429 uploaders. Sizes run from 2M to 20.4B parameters. The longest trained context is 128K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
SM
TwIL LM3
SmolLM · webAI-Official
Params3.08B
Context64K
Q4_K_M1.9 GB
Reasoning
SM
SmolLM2 135M Instruct
SmolLM · unsloth
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
SM
SmolLM2 135M Instruct
SmolLM · bartowski
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
SM
SmolLM3 3B
SmolLM · bartowski
Params3.08B
Context64K
Q4_K_M1.9 GB
GGUF
SM
SmolLM3 3B
SmolLM · ggml-org
Params3.08B
Context64K
Q4_K_M1.9 GB
GGUF
SM
smollm2 360m aac watch
SmolLM · dcostenco
Params362M
Context8K
Smallest0.2 GB
GGUF
SM
SmolLM2 360M Instruct
SmolLM · bartowski
Params362M
Context8K
Q4_K_M0.3 GB
GGUF
SM
SmolLM2 360M Instruct
SmolLM · HuggingFaceTB
Params362M
Context8K
Smallest0.4 GB
GGUF
SM
SmolLM2 1.7B Instruct
SmolLM · bartowski
Params1.71B
Context8K
Q4_K_M1.1 GB
GGUF
SM
Smollm3-3B
SmolLM · unsloth
Params3.08B
Context64K
Q4_K_M1.9 GB
GGUF
SM
SmolLM2 1.7B Instruct
SmolLM · HuggingFaceTB
Params1.71B
Context8K
Q4_K_M1.1 GB
GGUF
SM
SmolLM2 135M Instruct
SmolLM · mradermacher
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
SM
SmolLM2 135M
SmolLM · QuantFactory
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
SM
SmolLM2 135M
SmolLM · prithivMLmods
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
SM
Smollm3-3B-128K
SmolLM · unsloth
Params3.08B
Context128K
Q4_K_M1.9 GB
GGUF
SM
TwIL LM
SmolLM · webAI-Official
Params1.71B
Context8K
Q4_K_M1.1 GB
Reasoning
SM
SmolLM 1.7B
SmolLM · bartowski
Params1.71B
Context2K
Q4_K_M1.1 GB
Tool use
SM
SmolLM2 1.7B Instruct
SmolLM · lmstudio-community
Params1.71B
Context8K
Q4_K_M1.1 GB
GGUF
SM
SmolLM 135M
SmolLM · QuantFactory
Params135M
Context2K
Q4_K_M0.1 GB
GGUF
SM
SmolLM2 135M Instruct
SmolLM · lmstudio-community
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
SM
KrynexAI 1.4
SmolLM · KrynexLabs
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
SM
nisten/Biggie-SmoLlm-0.15B-Base
SmolLM · nisten
Params152M
Context2K
Smallest0.2 GB
GGUF
SM
SmolLM2 360M
SmolLM · QuantFactory
Params362M
Context8K
Q4_K_M0.3 GB
GGUF
SM
SmolLM 135M
SmolLM · HuggingFaceTB
Params135M
Context2K
Smallest0.1 GB
Tool use
SM
SmolLM2 135M Instruct
SmolLM · QuantFactory
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
SM
SmolLM2 360M Instruct
SmolLM · unsloth
Params362M
Context8K
Q4_K_M0.3 GB
GGUF
SM
Biggie SmoLlm 0.15B Base
SmolLM · mradermacher
Params181M
Context2K
Q4_K_M0.1 GB
GGUF
SM
SmolLM 360M
SmolLM · Felladrin
Params362M
Context2K
Smallest0.2 GB
GGUF
SM
SmolLM2-135M
SmolLM · ggml-org
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
SM
SmolLM 360M Instruct
SmolLM · mradermacher
Params362M
Context2K
Q4_K_M0.3 GB
Tool use

About the SmolLM GGUF models

SmolLM is represented on ggufmodels.com by 1,353 GGUF models, quantized by 429 uploaders on Hugging Face. The most downloaded are TwIL LM3, SmolLM2 135M Instruct and SmolLM2 135M Instruct. Parameter counts run from 2M to 20.4B, and the longest trained context window is 128K tokens.

A Q4_K_M file of a SmolLM model is between 0.0 to 14.1 GB, which is the quant most people start with. Most files come from RichardErkhov (280), mradermacher (248) and tensorblock (19). The quants you will meet most often are Q4_K_M (961), Q8_0 (863), Q5_K_M (707) and Q6_K (671), where the number is how many models offer that quant.

Frequently asked questions

Which SmolLM GGUF should I download?

The most downloaded SmolLM GGUF here is TwIL LM3. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a SmolLM model need?

Q4_K_M files of SmolLM models on this site range from 0.0 to 14.1 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do SmolLM GGUF files come from?

The most common quantizers here are RichardErkhov, mradermacher and tensorblock. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing