Orion GGUF models

70 GGUF models in the Orion family, quantized by 19 uploaders. Sizes run from 1.79B to 25.2B parameters. The longest trained context is 256K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
OR
Orion 26B A4B v1.1
Orion · bartowski
Params25.2B
Context256K
Q4_K_M18.0 GB
Vision
OR
Orion 26B A4B v1
Orion · bartowski
Params25.2B
Context256K
Q4_K_M17.0 GB
Vision
OR
Orion Kitchoon 26B A4B
Orion · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
ReasoningRoleplayTool use
OR
Orion 26B A4B v1.1
Orion · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
GGUF
OR
Orion-14B-Chat-0126-ck19800
Orion · OrionStarAI
Params14.5B
Context4K
Smallest29.0 GB
Coding
OR
Orion 9B Agentic CodeCore Merge
Orion · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningTool use
OR
Orion 26B A4B v1
Orion · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
GGUF
OR
Orion 26B A4B v1.1
Orion · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
Vision
OR
Orion 26B A4B v1
Orion · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
Vision
OR
Orion Kitchoon 26B A4B
Orion · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
ReasoningRoleplayTool use
OR
Orion 14B Base
Orion · mradermacher
Params14.5B
Context4K
Q4_K_M8.8 GB
Coding
OR
Orion 26B A4B v1h
Orion · BeaverAI
Params25.2B
Context256K
Q4_K_M16.8 GB
ReasoningTool use
OR
Orion 9B Agentic CodeCore Merge
Orion · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningTool use
OR
Orion 14B LongChat
Orion · mradermacher
Params14.5B
Context195K
Q4_K_M8.8 GB
Coding
OR
Orion 14B Chat
Orion · mradermacher
Params14.5B
Context4K
Q4_K_M8.8 GB
Coding
OR
Orion 9B Agentic CodeCore Merge
Orion · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningTool use
OR
Orion 26B A4B v1.1
Orion · K4r14
Params25.2B
Context256K
Smallest18.5 GB
ReasoningTool use
OR
Orion 26B A4B v1g
Orion · BeaverAI
Params25.2B
Context256K
Q4_K_M16.8 GB
ReasoningTool use
OR
Orion v1 8B
Orion · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
OR
Orion 14B Chat RAG
Orion · mradermacher
Params14.5B
Context4K
Q4_K_M8.8 GB
Coding
OR
Orion 14B Base Special
Orion · mradermacher
Params14.5B
Context4K
Q4_K_M8.8 GB
GGUF
OR
Orion 26B A4B v1b
Orion · BeaverAI
Params25.2B
Context256K
Q4_K_M16.8 GB
ReasoningTool use
OR
Orion 14B Base
Orion · mradermacher
Params14.5B
Context4K
Q4_K_M8.8 GB
Coding
OR
L3.1 Orion a 8B i1
Orion · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
OR
Orion 14B Chat Plugin
Orion · mradermacher
Params14.5B
Context4K
Q4_K_M8.8 GB
Coding
OR
Orion HYMT1.5 7B SFT v2601
Orion · mradermacher
Params7.5B
Context256K
Q4_K_M4.6 GB
GGUF
OR
L3.1 Orion b 8B i1
Orion · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
OR
Orion 14B Chata KoAlu2k
Orion · mradermacher
Params14.5B
Context4K
Q4_K_M8.8 GB
GGUF
OR
Orion 26B A4B v1a
Orion · BeaverAI
Params25.2B
Context256K
Q4_K_M16.8 GB
ReasoningTool use
OR
Orion 14B Chat Plugin
Orion · mradermacher
Params14.5B
Context4K
Q4_K_M8.8 GB
Coding

About the Orion GGUF models

Orion is represented on ggufmodels.com by 70 GGUF models, quantized by 19 uploaders on Hugging Face. The most downloaded are Orion 26B A4B v1.1, Orion 26B A4B v1 and Orion Kitchoon 26B A4B. Parameter counts run from 1.79B to 25.2B, and the longest trained context window is 256K tokens.

A Q4_K_M file of a Orion model is between 1.6 to 18.0 GB, which is the quant most people start with. Most files come from mradermacher (32), BeaverAI (8) and demonsu (5). The quants you will meet most often are Q6_K (52), Q4_K_M (50), Q2_K (48) and Q5_K_M (47), where the number is how many models offer that quant.

Frequently asked questions

Which Orion GGUF should I download?

The most downloaded Orion GGUF here is Orion 26B A4B v1.1. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Orion model need?

Q4_K_M files of Orion models on this site range from 1.6 to 18.0 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Orion GGUF files come from?

The most common quantizers here are mradermacher, BeaverAI and demonsu. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing