Orca GGUF models

161 GGUF models in the Orca family, quantized by 36 uploaders. Sizes run from 1.1B to 72.7B parameters. The longest trained context is 256K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
OR
Orca 2 13b
Orca · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
OR
Orca 2 7B
Orca · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
OR
Orca 2 13B
Orca · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
OR
Orca2myth7.2
Orca · TheBloke
Params20B
Context4K
Q4_K_M12.0 GB
GGUF
OR
Orca 2 13b
Orca · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
OR
Orca 2 13B SFT v5
Orca · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
Coding
OR
Orca 2 13b
Orca · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
OR
orca mini 13b i1
Orca · mradermacher
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
OR
models--mergekit-community--mergekit-ties-fnacfof
Orca · MaziyarPanahi
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
OR
Orca 2 13b
Orca · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
OR
Orca 2 13b
Orca · QuantFactory
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
OR
Dolphin 4B Instruct 0409
Orca · mradermacher
Params4B
Context16K
Q4_K_M2.5 GB
GGUF
OR
Orca Agent v0.1
Orca · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
CodingTool use
OR
Rhea 4B Coding Max
Orca · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
CodingReasoning
OR
Orca 2 13b SFT v6
Orca · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
Coding
OR
Orca 2 13B
Orca · second-state
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
OR
orca mini v3 13B
Orca · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
OR
orca mini v4 8b i1
Orca · mradermacher
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
OR
orca mini v3 7B
Orca · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
OR
Orca 2 13B No_Robots
Orca · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
Coding
OR
orca mini v3 70B
Orca · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
OR
Rhea 4B Coding Max
Orca · mradermacher
Params4.02B
Context256K
Q4_K_M2.5 GB
CodingReasoning
OR
orca mini v8 0 70b
Orca · bartowski
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
OR
Rhea 4B fast 0409 high i1
Orca · mradermacher
Params4B
Context32K
Q4_K_M2.5 GB
CodingReasoning
OR
Dolphin 4B Instruct 0409
Orca · mradermacher
Params4.02B
Context16K
Q4_K_M2.5 GB
Tool use
OR
Orca Agent v0.1
Orca · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
CodingTool use
OR
orca mini v8 0 70b i1
Orca · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
OR
orca mini 7b i1
Orca · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
OR
orca mini v8 1 70b
Orca · bartowski
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
OR
Orca Hermes 7B slerp i1
Orca · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use

About the Orca GGUF models

Orca is represented on ggufmodels.com by 161 GGUF models, quantized by 36 uploaders on Hugging Face. The most downloaded are Orca 2 13b, Orca 2 7B and Orca 2 13B. Parameter counts run from 1.1B to 72.7B, and the longest trained context window is 256K tokens.

A Q4_K_M file of a Orca model is between 0.7 to 47.4 GB, which is the quant most people start with. Most files come from mradermacher (90), tensorblock (15) and RichardErkhov (10). The quants you will meet most often are Q2_K (132), Q4_K_M (122), Q3_K_M (121) and Q5_K_M (118), where the number is how many models offer that quant.

Frequently asked questions

Which Orca GGUF should I download?

The most downloaded Orca GGUF here is Orca 2 13b. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Orca model need?

Q4_K_M files of Orca models on this site range from 0.7 to 47.4 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Orca GGUF files come from?

The most common quantizers here are mradermacher, tensorblock and RichardErkhov. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing