Openchat GGUF models

128 GGUF models in the Openchat family, quantized by 40 uploaders. Sizes run from 7M to 13B parameters. The longest trained context is 128K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
OP
openchat-3.6-8b-20240522
Openchat · legraphista
Params8B
Context8K
Smallest2.0 GB
GGUF
OP
openchat_openchat-3.5-1210
Openchat · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
OP
openchat_openchat_3.5
Openchat · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
openchat_openchat-3.5-0106
Openchat · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
weyaxi_openchat-3.5-1210-seraph-slerp
Openchat · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
openchat-3.5-0106
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
OpenChat 3.5 7B Starling v2.0
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
openchat 3.5 i1
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
OP
OpenChat 3.5 7B SOLAR v2.0 i1
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
Openchat_3.5
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
OP
OpenChat 3.5 7B Starling v2.0
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
openchat 3.6 8b 20240522
Openchat · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
OP
OpenChat 3.5 7B SOLAR v2.0
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
openchat 3.5 1210 Seraph Slerp
Openchat · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
openchat v3.2 super
Openchat · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
OP
openchat-3.6-8b-20240522
Openchat · crusoeai
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
OP
openchat-3.5-0106
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
openchat 3.5 1210 i1
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
OP
Openchat 3.5 0106
Openchat · mradermacher
Params10.7B
Context8K
Q4_K_M6.5 GB
GGUF
OP
sanjiwatsuki_openchat-3.5-1210-starling-slerp
Openchat · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
openchat-3.5-0106
Openchat · duyntnet
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
Openchat 3.5 0106
Openchat · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
Openchat_3.5
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
OP
Openchat_3.5
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
OP
Openchat 3.5 0106
Openchat · RichardErkhov
Params10.7B
Context8K
Q4_K_M6.5 GB
GGUF
OP
openchat 3.6 8b 20240522
Openchat · mradermacher
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
OP
Openchat 3.5 0106
Openchat · mradermacher
Params8.99B
Context8K
Q4_K_M5.4 GB
GGUF
OP
openchat-3.6-8b-20240522
Openchat · duyntnet
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
OP
openchat-3.5-0106-11b
Openchat · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
OP
Openchat 3.5 0106
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF

About the Openchat GGUF models

Openchat is represented on ggufmodels.com by 128 GGUF models, quantized by 40 uploaders on Hugging Face. The most downloaded are openchat-3.6-8b-20240522, openchat_openchat-3.5-1210 and openchat_openchat_3.5. Parameter counts run from 7M to 13B, and the longest trained context window is 128K tokens.

A Q4_K_M file of a Openchat model is between 4.4 to 7.9 GB, which is the quant most people start with. Most files come from mradermacher (43), RichardErkhov (11) and tensorblock (10). The quants you will meet most often are Q4_K_M (96), Q2_K (95), Q5_K_M (89) and Q6_K (86), where the number is how many models offer that quant.

Frequently asked questions

Which Openchat GGUF should I download?

The most downloaded Openchat GGUF here is openchat-3.6-8b-20240522. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Openchat model need?

Q4_K_M files of Openchat models on this site range from 4.4 to 7.9 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Openchat GGUF files come from?

The most common quantizers here are mradermacher, RichardErkhov and tensorblock. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing