GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
Phi 3 mini 4k instruct GGUF v3
Phi · QuantFactory
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
QW
Apollo2 7B
Qwen · QuantFactory
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
LL
LLaMA O1 Base 1127
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
SA
Sarvam 2b v0.5
Sarvam · QuantFactory
Params2.51B
Context8K
Q4_K_M1.5 GB
GGUF
ES
EstopianMaid 13B
Estopianmaid · QuantFactory
Params13B
Context4K
Q4_K_M7.9 GB
Roleplay
PR
Promt generator
Promt · QuantFactory
Params816M
Context2K
Q4_K_M0.6 GB
GGUF
LL
opus v1.2 llama 3 8b
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Tess 2.0 Llama 3 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
FI
Fimbulvetr 11B v2
Fimbulvetr · QuantFactory
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
CA
CabraMistral v3 7b 32k
Cabramistral · QuantFactory
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
TA
TAIDE LX 7B Chat
Taide · QuantFactory
Params6.94B
Context4K
Q4_K_M4.2 GB
GGUF
LL
Llama 3.1 8B ArliAI RPMax v1.2
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
NO
Nous Hermes 13b Chinese
Nous · QuantFactory
Params13.2B
Context2K
Q4_K_M8.0 GB
Tool use
AR
ArrowPro 7B KUJIRA
Arrowpro · QuantFactory
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
LL
MagpieLM 8B SFT v0.1
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
CA
CS Calme Instruct 7b
Calme · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Thespis Llama 3.1 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Roleplay
LL
Llama 3.1 8B ArliAI Indo Formax v1.0
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
gemma 7b
Gemma · QuantFactory
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
GE
Gemma 2 2b It
Gemma · QuantFactory
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
LL
BlackSheep Llama3.2 3B
Llama · QuantFactory
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
LL
Llama 3.1 Hawkish 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
deepthought 8b llama v0.01 alpha
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Meta Llama 3 120B Instruct
Llama · QuantFactory
Params122B
Context8K
Q4_K_M73.2 GB
GGUF
MI
Wayfarer 2 12B
Mistral · QuantFactory
Params12.2B
Context128K
Q4_K_M7.5 GB
Roleplay
WR
writing roleplay 20k context nemo 12b v1.0
Writing · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Roleplay
LL
llm compiler 13b ftd
Llm · QuantFactory
Params13B
Context16K
Q4_K_M7.9 GB
GGUF
NA
NaturalLM
Naturallm · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
MI
math shepherd mistral 7b rl
Mistral · QuantFactory
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
QW
qwen2.5 7b ins v3
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF

Keep browsing