Gpt GGUF models

724 GGUF models in the Gpt family, quantized by 143 uploaders. Sizes run from 2M to 117B parameters. The longest trained context is 256K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
GP
Gpt2 Medium
Gpt · mradermacher
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
GP
gpt2 GGUF smashed
Gpt · PrunaAI
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
GP
gpt2 i1
Gpt · mradermacher
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
GP
GPT4 X Alpasta 30b i1
Gpt · mradermacher
Params32.5B
Context2K
Q4_K_M19.6 GB
GGUF
GP
gpt2
Gpt · mradermacher
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
GP
Gpt Neox 20b
Gpt · mradermacher
Params20.6B
Context2K
Q4_K_M13.1 GB
GGUF
GP
gpt2
Gpt · QuantFactory
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
GP
Gpt2 Medium
Gpt · mradermacher
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
GP
GPT2-774M-CINDER-SHOW-MULTI-CHAT
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
GP
Gpt2 Large
Gpt · mradermacher
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
GP
Gpt2 Large
Gpt · mradermacher
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
GP
GPT4chan 8B
Gpt · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GP
openai community gpt2 GGUF smashed
Gpt · PrunaAI
Params124M
Context1K
Q4_K_M0.1 GB
GGUF
GP
Gpt2 Xl
Gpt · mradermacher
Params1.64B
Context1K
Q4_K_M1.2 GB
GGUF
GP
gpt 5.6 sol undecillion 6u5go5
Gpt · oroboros-labs
Params27.3B
Context256K
Smallest0.9 GB
GGUF
GP
Gpt Neox 20b
Gpt · mradermacher
Params20.6B
Context2K
Q4_K_M13.1 GB
GGUF
GP
gpt astra 7 undecillion 7u5go5
Gpt · oroboros-labs
Params27.3B
Context—
Smallest0.9 GB
GGUF
GP
GPT4 X Alpasta 30b
Gpt · mradermacher
Params32.5B
Context2K
Q4_K_M19.6 GB
GGUF
GP
gpt 5.6 sol
Gpt · oroboros-labs
Params8.19B
Context128K
Smallest5.0 GB
Tool use
GP
GPT4chan 8B
Gpt · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GP
GPT4chan 24B
Gpt · bartowski
Params23.6B
Context32K
Q4_K_M14.3 GB
GGUF
GP
gpt4 x alpaca i1
Gpt · mradermacher
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
GP
Gpt4O Distil Paperwitch Abliteration L33 70b
Gpt · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
GP
gpt2-small-danish
Gpt · afrideva
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
GP
GPT4 x AlpacaDente 30b
Gpt · mradermacher
Params32.5B
Context2K
Q4_K_M19.6 GB
GGUF
GP
gpt2-xl
Gpt · RichardErkhov
Params1.64B
Context1K
Q4_K_M1.2 GB
GGUF
GP
Yuuki best
Gpt · OpceanAI
Params82M
Context1K
Q4_K_M0.1 GB
Coding
GP
Openai Community.Gpt2 Xl
Gpt · DevQuasar-10
Params1.64B
Context1K
Q4_K_M1.2 GB
GGUF
GP
Gpt Nyc
Gpt · mradermacher
Params356M
Context1K
Q4_K_M0.2 GB
GGUF
GP
Gpt2 Xl
Gpt · mradermacher
Params1.64B
Context1K
Q4_K_M1.2 GB
GGUF

About the Gpt GGUF models

Gpt is represented on ggufmodels.com by 724 GGUF models, quantized by 143 uploaders on Hugging Face. The most downloaded are Gpt2 Medium, gpt2 GGUF smashed and gpt2 i1. Parameter counts run from 2M to 117B, and the longest trained context window is 256K tokens.

A Q4_K_M file of a Gpt model is between 0.0 to 87.9 GB, which is the quant most people start with. Most files come from mradermacher (286), RichardErkhov (153) and tensorblock (65). The quants you will meet most often are Q2_K (537), Q4_K_M (534), Q3_K_M (488) and Q5_K_M (487), where the number is how many models offer that quant.

Frequently asked questions

Which Gpt GGUF should I download?

The most downloaded Gpt GGUF here is Gpt2 Medium. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Gpt model need?

Q4_K_M files of Gpt models on this site range from 0.0 to 87.9 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Gpt GGUF files come from?

The most common quantizers here are mradermacher, RichardErkhov and tensorblock. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing