GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MA
magnum v4 12b
Magnum · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
MI
Mistral 7B Instruct v0.1
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
NI
B NIMITA L3 8B v0.02
Nimita · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
Qwen2.5 14B Instruct 1M
Qwen · QuantFactory
Params14.8B
Context986K
Q4_K_M9.0 GB
Tool use
DA
Daredevil 8B
Daredevil · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Hermes 3 Llama 3.2 3b Base Fft Chatml Rerun 8xh100 E4 2024 11 24 4k Steps
Llama · QuantFactory
Params3.61B
Context128K
Q4_K_M2.2 GB
RoleplayTool use
LL
Llama 3.1 Literotica 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
BI
Bitnet_B1_58 3B
Bitnet · QuantFactory
Params3.32B
Context2K
Q4_K_M2.5 GB
GGUF
BE
CausalLM 35b beta long
Beta · QuantFactory
Params35B
Context8K
Q4_K_M21.5 GB
Tool use
LL
Llama 3.1 8B ArliAI RPMax v1.1
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen 2.5 3b Evol CoT
Qwen · QuantFactory
Params3.09B
Context32K
Q4_K_M1.9 GB
RoleplayTool use
AZ
Azure Dusk v0.2
Azure · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Roleplay
LL
Llama 3.1 8B ArliAI RPMax v1.0
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MA
mamba 2.8b hf
Mamba · QuantFactory
Params2.77B
Context1024K
Q4_K_M1.6 GB
GGUF
PH
Flow Judge v0.1
Phi · QuantFactory
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
QW
OpenR1-Qwen-7B
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
ReasoningTool use
MI
Ministral 8B Instruct 2410
Mistral · QuantFactory
Params8.02B
Context32K
Q4_K_M4.9 GB
Tool use
QW
li 14b v0.4
Qwen · QuantFactory
Params1.78B
Context128K
Q4_K_M1.1 GB
CodingReasoningTool use
LL
Llama 3 8B 16K
Llama · QuantFactory
Params8.03B
Context16K
Q4_K_M4.9 GB
GGUF
LL
Tulu 3.1 8B SuperNova
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
SA
sabia 7b
Sabia · QuantFactory
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
LL
Llama 3.1 Tulu 3 8B SFT
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
Dante 9B
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
SQ
sqlcoder 7b 2
Sqlcoder · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
NE
Neeto 1.0 8b
Neeto · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LY
MN 12B Lyra v3
Lyra · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
ST
starling 7b raft ft
Starling · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
Jan nano
Qwen · QuantFactory
Params4.41B
Context40K
Q4_K_M2.7 GB
ReasoningTool use
TW
TwinLlama 3.1 8B
Twinllama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
AI Sweden Llama 3 8B Instruct
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use

Keep browsing