GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
Phi 3 mini 4k geminified
Phi · QuantFactory
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
LL
Llama3 German 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3 Alpha Ko 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
TI
Tinyllama 15M
Tinyllama · QuantFactory
Params15M
Context256
Q4_K_M0.0 GB
GGUF
GR
granite 3b code base 128k
Granite · QuantFactory
Params3.48B
Context125K
Q4_K_M2.1 GB
Coding
BA
internistai base 7b v0.2
Base · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
Starling LM 7B beta
Starling · QuantFactory
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
ST
Starcoder2 3b
StarCoder · QuantFactory
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
PH
Phi 3 mini 4k instruct
Phi · QuantFactory
Params3.82B
Context4K
Q4_K_M2.3 GB
Coding
AU
AutoCoder S 6.7B
Autocoder · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Llama 3.2 3B Agent007 Coder
Llama · QuantFactory
Params3.21B
Context128K
Q4_K_M2.0 GB
Coding
QW
Dolphin3.0 Qwen2.5 3B
Qwen · QuantFactory
Params3.4B
Context32K
Q4_K_M2.1 GB
CodingTool use
YI
Yi 1.5 6B Chat
Yi · QuantFactory
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
HE
HelpingAI 3B coder
Helpingai · QuantFactory
Params3.48B
Context125K
Q4_K_M2.1 GB
Coding
QW
Qwen3 Reranker 8B
Qwen · QuantFactory
Params8.2B
Context40K
Q4_K_M5.0 GB
Embedding
LL
Fireball Meta Llama 3.2 8B Instruct agent 003 128k code DPO
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
Coding
LL
Dolphin 3.0 Llama 3.2 3B
Llama · QuantFactory
Params3.61B
Context128K
Q4_K_M2.2 GB
CodingTool use
NO
notus 7b v1
Notus · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 8B TKK Elite V1.0
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GE
gemma 2 9b it
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
LongWriter Llama3.1 8b
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 14B_Uncencored_Instruct
Qwen · QuantFactory
Params14.8B
Context128K
Q4_K_M9.0 GB
GGUF
ST
starcoderbase 7b
StarCoder · QuantFactory
Params7.5B
Context8K
Q4_K_M4.9 GB
Coding
LL
DarkIdol Llama 3.1 8B Instruct 1.0 Uncensored
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
DE
DeepSeek Coder V2 Lite Base
DeepSeek · QuantFactory
Params15.7B
Context160K
Q4_K_M10.4 GB
Coding
QW
SuperCorrect 7B
Qwen · QuantFactory
Params7.6B
Context4K
Q4_K_M4.7 GB
GGUF
LL
Dolphin 3.0 Llama 3.2 1b
Llama · QuantFactory
Params1.5B
Context128K
Q4_K_M1.0 GB
CodingTool use
MI
Mistral Trismegistus 7B
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 14B Instruct
Qwen · QuantFactory
Params14.8B
Context32K
Q4_K_M9.0 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · QuantFactory
Params3.82B
Context128K
Q4_K_M2.4 GB
Coding

Keep browsing