GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
CO
Control Nanuq 8B
Control · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
RoleplayTool use
GR
granite 3.1 8b instruct
Granite · QuantFactory
Params8.37B
Context128K
Q4_K_M5.1 GB
Tool use
TH
TherapyBeagle 11B v1
Therapybeagle · QuantFactory
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
MI
Lily Cybersecurity 7B v0.2
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Linkbricks Horizon AI Korean Advanced 12B
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
MI
dolphin 2.2.1 mistral 7b
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mistral nemo gutenberg 12B v3
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
Pantheon RP 1.0 8b Llama 3
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Roleplay
LF
LFM2 350M Math
Lfm · QuantFactory
Params422M
Context125K
Q4_K_M0.3 GB
GGUF
QW
Homunculus
Qwen · QuantFactory
Params12.5B
Context128K
Q4_K_M7.6 GB
ReasoningTool use
AY
aya expanse 8b
Aya · QuantFactory
Params8.03B
Context8K
Q4_K_M5.1 GB
GGUF
AV
Average Normie v3.69 8B
Average · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
Quill v1
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
Mistral 7B Instruct v0.1
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 Nephilim v3 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
UN
UNfilteredAI 1B
Unfilteredai · QuantFactory
Params1.06B
Context2K
Q4_K_M0.6 GB
GGUF
QW
Qwen2 1.5B
Qwen · QuantFactory
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
GR
granite 3.1 1b a400m instruct
Granite · QuantFactory
Params1.38B
Context128K
Q4_K_M0.9 GB
Tool use
LL
Lama DPOlphin 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 7B HomerCreative Mix
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
RoleplayTool use
QW
Qwen2.5 Coder 0.5B Instruct
Qwen · QuantFactory
Params630M
Context32K
Q4_K_M0.5 GB
CodingTool use
LL
Romulus cpt Llama 3.1 8B v0.1
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
Gemma 2 9b It
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
Tool use
CH
MN 12B Chronos Gold Celeste v1
Chronos · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
Llama 3 ELYZA JP 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IN
InternLM2
InternLM · QuantFactory
Params7.74B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Tulu 3.1 8B SuperNova
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
KingNish Llama3 8b
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 Coder 14B Instruct
Qwen · QuantFactory
Params14.8B
Context32K
Q4_K_M9.0 GB
CodingTool use

Keep browsing