GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
RO
MN 12b Sunrose
Rosier · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
GR
granite 8b code instruct 128k
Granite · QuantFactory
Params8.05B
Context125K
Q4_K_M4.9 GB
CodingTool use
HO
Holland Magnum Merge
Holland · QuantFactory
Params4.51B
Context8K
Q4_K_M2.8 GB
GGUF
MI
Minitron 8B Base
Minitron · QuantFactory
Params8.27B
Context4K
Q4_K_M5.2 GB
GGUF
QW
UwU 7B Instruct
Qwen · QuantFactory
Params7.62B
Context128K
Q4_K_M4.7 GB
ReasoningTool use
SH
shieldgemma 9b
Shieldgemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
KONI Llama3 8B 20240630
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MA
Matter 0.2 7B DPO
Matter · QuantFactory
Params7B
Context32K
Q4_K_M4.4 GB
GGUF
SA
Saiga_Tlite_8B_Abliterated_Sft_M1_D9
Saiga · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
TU
turn detector
Turn · QuantFactory
Params163M
Context8K
Q4_K_M0.1 GB
GGUF
JA
L3 Scrambled Eggs On Toast 8B
Jamet · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Roleplay
DI
Distilgpt2
Distilgpt · QuantFactory
Params121M
Context1K
Q4_K_M0.1 GB
Coding
YI
Yi 1.5 9B
Yi · QuantFactory
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
Mistral 7B v0.1
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2 7B BD RP
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
Roleplay
MI
HolyNemo 12B
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
SM
NuExtract 1.5 Smol
SmolLM · QuantFactory
Params1.81B
Context8K
Q4_K_M1.1 GB
GGUF
CR
Crimson Dawn V0.1
Crimson · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
NY
Nyanade Stunna Maid 7B v0.2
Nyanade · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
TIPO 200M
Tipo · QuantFactory
Params203M
Context2K
Q4_K_M0.1 GB
Vision
QW
BFS Prover
Qwen · QuantFactory
Params7.62B
Context4K
Q4_K_M4.7 GB
Tool use
LL
Llama 3 MAAL 8B Instruct v0.1
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GR
granite 3.0 1b a400m base
Granite · QuantFactory
Params1.33B
Context4K
Q4_K_M0.8 GB
GGUF
QW
Qwen2 7B Instruct
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen2.5 Coder 7B
Qwen · QuantFactory
Params7.61B
Context32K
Q4_K_M4.7 GB
CodingTool use
LL
L3 Umbral Mind RP v2.0 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Roleplay
GE
Tiger Gemma 9B v3
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
mistral nemo bophades 12B
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
LL
Llama 3 KoEn 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Keep browsing