OLMo GGUF models

756 GGUF models in the OLMo family, quantized by 151 uploaders. Sizes run from 7M to 32.2B parameters. The longest trained context is 128K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
OL
allenai_olmOCR 2 7B 1025
OLMo · lmstudio-community
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
OL
olmOCR 2 7B 1025
OLMo · bartowski
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
OL
OLMo 2 1124 13B Instruct
OLMo · bartowski
Params13.7B
Context4K
Q4_K_M8.4 GB
GGUF
OL
OLMoE 1B 7B 0924 Instruct
OLMo · bartowski
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
OL
Olmo-3-7B-Instruct
OLMo · unsloth
Params7.3B
Context64K
Q4_K_M4.5 GB
GGUF
OL
Olmo 3.1 32B Instruct
OLMo · bartowski
Params32.2B
Context64K
Q4_K_M19.5 GB
GGUF
OL
OLMoE 1B 7B 0924 Instruct
OLMo · mradermacher
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
OL
Olmo-3-7B-Think
OLMo · unsloth
Params7.3B
Context64K
Q4_K_M4.5 GB
Reasoning
OL
OLMoE 1B 7B 0924 Instruct
OLMo · allenai
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
OL
Olmo 3.1 32B Think
OLMo · bartowski
Params32.2B
Context64K
Q4_K_M19.5 GB
Reasoning
OL
OLMo 7B 0424 Hf
OLMo · mradermacher
Params6.89B
Context4K
Q4_K_M4.2 GB
GGUF
OL
OLMoE 1B 7B 0125 Instruct
OLMo · allenai
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
OL
OLMo 2 1124 7B Instruct
OLMo · bartowski
Params7.3B
Context4K
Q4_K_M4.5 GB
GGUF
OL
Olmo-3-1025-7B
OLMo · Mungert
Params7.3B
Context64K
Q4_K_M4.7 GB
GGUF
OL
Olmo-3-32B-Think
OLMo · unsloth
Params32.2B
Context64K
Q4_K_M19.5 GB
Reasoning
OL
AMD OLMo 1B SFT DPO
OLMo · lmstudio-community
Params1.18B
Context2K
Q4_K_M0.7 GB
GGUF
OL
OLMoE 1B 7B 0924 Instruct
OLMo · mradermacher
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
OL
OLMo-1.7-7B-hf
OLMo · mradermacher
Params6.89B
Context4K
Q4_K_M4.2 GB
GGUF
OL
OLMo 1B 0724 Hf
OLMo · mradermacher
Params1.28B
Context4K
Q4_K_M0.8 GB
GGUF
OL
Olmo-3.1-32B-Instruct
OLMo · unsloth
Params32.2B
Context64K
Q4_K_M19.5 GB
GGUF
OL
OLMo 2 0425 1B Instruct
OLMo · allenai
Params1.48B
Context4K
Q4_K_M0.9 GB
GGUF
OL
Olmo 3 Think AIO
OLMo · prithivMLmods
Params32.2B
Context64K
Q4_K_M4.5 GB
Reasoning
OL
Olmo 3 7B Instruct DPO
OLMo · mradermacher
Params7.3B
Context64K
Q4_K_M4.5 GB
Reasoning
OL
Olmo-3-7B-Instruct
OLMo · Mungert
Params7.3B
Context64K
Q4_K_M4.7 GB
Tool use
OL
Olmo 3 7B Think
OLMo · mradermacher
Params7.3B
Context64K
Q4_K_M4.5 GB
Reasoning
OL
Olmo 3 7B Instruct SFT
OLMo · mradermacher
Params7.3B
Context64K
Q4_K_M4.5 GB
Tool use
OL
OLMo 2 1124 7B Instruct
OLMo · allenai
Params7.3B
Context4K
Q4_K_M4.5 GB
GGUF
OL
Olmo 3 7B Instruct
OLMo · bartowski
Params7.3B
Context64K
Q4_K_M4.5 GB
GGUF
OL
OLMoE-1B-7B-0125-Instruct-Distill-ot114k-batch32
OLMo · mradermacher
Params6.9B
Context4K
Q4_K_M4.2 GB
Reasoning
OL
OLMoE 1B 7B 0924 SFT
OLMo · mradermacher
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF

About the OLMo GGUF models

OLMo is represented on ggufmodels.com by 756 GGUF models, quantized by 151 uploaders on Hugging Face. The most downloaded are allenai_olmOCR 2 7B 1025, olmOCR 2 7B 1025 and OLMo 2 1124 13B Instruct. Parameter counts run from 7M to 32.2B, and the longest trained context window is 128K tokens.

A Q4_K_M file of a OLMo model is between 0.0 to 19.5 GB, which is the quant most people start with. Most files come from mradermacher (367), tensorblock (31) and RichardErkhov (25). The quants you will meet most often are Q4_K_M (597), Q6_K (533), Q2_K (530) and Q5_K_M (509), where the number is how many models offer that quant.

Frequently asked questions

Which OLMo GGUF should I download?

The most downloaded OLMo GGUF here is allenai_olmOCR 2 7B 1025. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a OLMo model need?

Q4_K_M files of OLMo models on this site range from 0.0 to 19.5 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do OLMo GGUF files come from?

The most common quantizers here are mradermacher, tensorblock and RichardErkhov. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing