GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama 3 8B Instruct Gradient 1048k GGUF v2
Llama · QuantFactory
Params8B
Context1024K
Q4_K_M4.9 GB
GGUF
LL
Llama 3.1 8B EZO 1.1 it
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
JA
Jais Adapted 7b Chat
Jais · QuantFactory
Params7B
Context4K
Q4_K_M4.3 GB
Coding
AL
Alpha Orionis v0.1
Alpha · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MY
MythoMax L2 13b
Mythomax · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
TU
Turkcell LLM 7b v1
Turkcell · QuantFactory
Params7.38B
Context32K
Q4_K_M4.5 GB
GGUF
MI
Mika 7B
Mika · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
NI
L3.1 Celestial Stone 2x8B
Niitama · QuantFactory
Params13.7B
Context128K
Q4_K_M8.3 GB
GGUF
CE
Celeste 12B V1.6
Celeste · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
TI
TinySlime 1.1B Chat v1.0
Tinyslime · QuantFactory
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
ZI
Zion_Alpha
Zion · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
XL
xLAM 7b r
Xlam · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Meta Llama 3 8B Instruct function calling
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
SE
SeaLLM 7B v2.5
Seallm · QuantFactory
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
BI
Bielik 11B v2.3 Instruct
Bielik · QuantFactory
Params11.2B
Context32K
Q4_K_M6.7 GB
GGUF
TI
TinyLlama 1.1B Chat v0.6
Tinyllama · QuantFactory
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
CO
Yi Coder 9B Chat
Coder · QuantFactory
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
NU
NuminiLlama 3.1 8B
Numinillama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
DE
Nemotron Research Reasoning Qwen 1.5B
DeepSeek · QuantFactory
Params1.78B
Context128K
Q4_K_M1.1 GB
ReasoningTool use
SQ
sqlcoder
Sqlcoder · QuantFactory
Params15.8B
Context8K
Q4_K_M10.1 GB
Coding
GE
Gemma 2 2B Stheno Filtered
Gemma · QuantFactory
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
LL
Llama3 DocChat 1.0 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3 8B Instruct Finance RAG
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
OpenMath2 Llama3.1 8B
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MY
Maxtopia 13B
Mythomax · QuantFactory
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
IN
InternLM2
InternLM · QuantFactory
Params1.89B
Context8K
Q4_K_M1.2 GB
GGUF
LL
Llama3.1 8B Esper2
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
CodingTool use
WI
Wizard Vicuna 13B Uncensored
Wizard · QuantFactory
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
NE
NVIDIA Nemotron Nano 12B v2
Nemotron · QuantFactory
Params12B
Context—
Q4_K_M6.3 GB
GGUF
QW
calme-2.8-qwen2-7b
Qwen · QuantFactory
Params7.62B
Context128K
Q4_K_M4.7 GB
Tool use

Keep browsing