GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Meta Llama 3 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 7B Instruct abliterated v2
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
GGUF
GL
Glm 4 9b Chat
GLM · QuantFactory
Params9.4B
Context128K
Q4_K_M6.3 GB
GGUF
LL
Llama 3.2 3B Instruct abliterated
Llama · QuantFactory
Params3.2B
Context128K
Q4_K_M2.0 GB
GGUF
LL
DarkIdol Llama 3.1 8B Instruct 1.2 Uncensored
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
SM
SmolLM2 135M
SmolLM · QuantFactory
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
QW
Qwen2.5 7B Instruct Uncensored
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
GGUF
CO
Cotype Nano
Cotype · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
MI
Mistral 7B Instruct v0.3
Mistral · QuantFactory
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
PE
Peach 2.0 9B 8k Roleplay
Peach · QuantFactory
Params8.8B
Context16K
Q4_K_M5.3 GB
Roleplay
MI
Mistral Nemo Instruct 2407 abliterated
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
PH
Phi 3 mini 128k instruct
Phi · QuantFactory
Params3.82B
Context128K
Q4_K_M2.4 GB
Coding
NE
NeuralDaredevil 8B abliterated
Neuraldaredevil · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
ST
L3 8B Stheno v3.2
Stheno · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
SM
SmolLM 135M
SmolLM · QuantFactory
Params135M
Context2K
Q4_K_M0.1 GB
GGUF
LL
dolphin 2.9 llama3 8b
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
CodingTool use
MA
mamba 130m hf
Mamba · QuantFactory
Params129M
Context1024K
Q4_K_M0.1 GB
GGUF
ST
L3 8B Stheno v3.1
Stheno · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 Coder 7B
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
Coding
SM
SmolLM2 360M
SmolLM · QuantFactory
Params362M
Context8K
Q4_K_M0.3 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
SM
SmolLM2 135M Instruct
SmolLM · QuantFactory
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
MI
Mistral Nemo Instruct 2407
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
GE
gemma 2 27b it abliterated
Gemma · QuantFactory
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
AT
N ATLaS
Atlas · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Meta Llama 3 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
ST
stable code instruct 3b
Stable · QuantFactory
Params2.8B
Context16K
Q4_K_M1.7 GB
Coding
MI
dolphin 2.8 mistral 7b v02
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 Patronus Lynx 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GP
gpt2
Gpt · QuantFactory
Params163M
Context1K
Q4_K_M0.1 GB
GGUF

About QuantFactory on Hugging Face

QuantFactory has published 1,418 GGUF models that ggufmodels.com tracks, across families such as Llama, Qwen and Mistral. The most downloaded are Meta Llama 3 8B Instruct, Qwen2.5 7B Instruct abliterated v2 and Glm 4 9b Chat.

The quants QuantFactory publishes most often are Q4_K_M (1415), Q4_0 (1407), Q8_0 (1404) and Q6_K (1399). Their Q4_K_M files range from 0.0 to 134.8 GB. Parameter counts run from 9M to 225B, and the longest trained context window is 4194304K tokens.

Frequently asked questions

What GGUF models has QuantFactory published?

1,418 models are listed here, mainly from the Llama, Qwen and Mistral families. The most downloaded is Meta Llama 3 8B Instruct.

Which quants does QuantFactory publish?

The most common are Q4_K_M (1415 models), Q4_0 (1407 models), Q8_0 (1404 models) and Q6_K (1399 models).

Keep browsing