Model GGUF models

1010 GGUF models in the Model family, quantized by 754 uploaders. Sizes run from 13M to 69B parameters. The longest trained context is 1024K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
MO
model 007 70B
Model · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
MO
Model Compilation
Model · Coercer
Params14.3B
Context—
Q4_K_M9.7 GB
GGUF
MO
PJ 7B MODEL FINAL
Model · mradermacher
Params6.95B
Context4K
Q4_K_M4.2 GB
GGUF
MO
model
Model · stablediffussion002
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MO
Rp Model v5
Model · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningRoleplayTool use
MO
Rp model_E2B_v5.1
Model · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
ReasoningRoleplayTool use
MO
E Model V1
Model · mradermacher
Params7.4B
Context32K
Q4_K_M4.5 GB
Reasoning
MO
YM 12B Model Stock i1
Model · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
MO
model app
Model · CreativeHub008
Params318M
Context—
Smallest0.0 GB
GGUF
MO
r model
Model · hackint0sh
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
MO
my model v02
Model · Lilgarca
Params230M
Context4K
Smallest0.1 GB
CodingReasoningTool use
MO
cr-model
Model · tastypear
Params14.2B
Context32K
Q4_K_M9.2 GB
GGUF
MO
model1
Model · PeYing
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
MO
E Model V1
Model · mradermacher
Params7.38B
Context32K
Q4_K_M4.5 GB
Reasoning
MO
Model_Pmc_Gamma_0.05_Noise0.2_0.1_Epoch0
Model · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
GGUF
MO
Rp Model v5
Model · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningRoleplayTool use
MO
QL model
Model · ashwanDevEDU
Params3.21B
Context128K
Smallest6.4 GB
Tool use
MO
Model Kinship 1 2
Model · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MO
YM 12B Model Stock
Model · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
MO
Model 3
Model · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
Tool use
MO
Model_Pmc_Gamma_0.1_Noise0.2_0.2_Epoch0
Model · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
GGUF
MO
Model Merging
Model · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
MO
Model_Pmc_Gamma_0.05_Noise0.2_0.2_Epoch2
Model · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
GGUF
MO
model quran tafseer model GPT2
Model · mradermacher
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MO
Model_Pmc_Kl_0.1_Noise0.2_0.1_Epoch0
Model · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
GGUF
MO
Model_Pmc_Gamma_0.05_Noise0.2_0.1_Epoch1
Model · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
GGUF
MO
model attribution challenge gpt2
Model · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MO
Model_Pmc_Beta_0.05_Noise0.2_0.1_Epoch0
Model · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
GGUF
MO
model
Model · Ajex
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
MO
model
Model · altarthedeer
Params1.24B
Context128K
Smallest2.5 GB
Tool use

About the Model GGUF models

Model is represented on ggufmodels.com by 1,010 GGUF models, quantized by 754 uploaders on Hugging Face. The most downloaded are model 007 70B, Model Compilation and PJ 7B MODEL FINAL. Parameter counts run from 13M to 69B, and the longest trained context window is 1024K tokens.

A Q4_K_M file of a Model model is between 0.0 to 41.4 GB, which is the quant most people start with. Most files come from RichardErkhov (37), LucileFavero (26) and mradermacher (18). The quants you will meet most often are Q8_0 (543), Q4_K_M (498), Q5_K_M (229) and F16 (228), where the number is how many models offer that quant.

Frequently asked questions

Which Model GGUF should I download?

The most downloaded Model GGUF here is model 007 70B. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Model model need?

Q4_K_M files of Model models on this site range from 0.0 to 41.4 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Model GGUF files come from?

The most common quantizers here are RichardErkhov, LucileFavero and mradermacher. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing