GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
GE
gemma 2 2b it Flight Multi Turn V3 DPO
Gemma · QuantFactory
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
LL
Vikhr Llama3.1 8B Instruct R 21 09 24
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GE
gemma 2 9b
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
Llama 3.1 8B Stheno v3.4
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
ST
starcoder2 3b
StarCoder · QuantFactory
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
MA
MAmmoTH2 7B Plus
Mammoth · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OL
OLMoE 1B 7B 0924
OLMo · QuantFactory
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · QuantFactory
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
NO
Nous Hermes 2 SOLAR 10.7B
Nous · QuantFactory
Params10.7B
Context4K
Q4_K_M6.5 GB
Tool use
MA
MagpieLM 4B Chat v0.1
Magpielm · QuantFactory
Params4.51B
Context128K
Q4_K_M2.8 GB
GGUF
AH
Ahma 3B
Ahma · QuantFactory
Params3.63B
Context2K
Q4_K_M2.8 GB
GGUF
PY
pythia 12b
Pythia · QuantFactory
Params11.8B
Context2K
Q4_K_M7.6 GB
GGUF
MI
Hermes2 Gutenberg2 Mistral 7B
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MI
Mistral Nemo Base 2407
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
DE
DeepSeek Coder V2 Lite Instruct
DeepSeek · QuantFactory
Params15.7B
Context160K
Q4_K_M10.4 GB
Coding
MI
dolphin 2.1 mistral 7b
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 14B Gutenberg 1e Delta
Qwen · QuantFactory
Params14.8B
Context32K
Q4_K_M9.0 GB
GGUF
SO
KULLM3
Solar · QuantFactory
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
QW
Qwen2.5 Sex
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
QW
Qwen2.5 Math 7B
Qwen · QuantFactory
Params7.6B
Context4K
Q4_K_M4.7 GB
GGUF
HE
HelpingAI 9B
Helpingai · QuantFactory
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
LL
Dorna Llama3 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Codestral 22B v0.1
Mistral · QuantFactory
Params22.2B
Context32K
Q4_K_M13.3 GB
Coding
VI
MN Violet Lotus 12B
Violet · QuantFactory
Params12.2B
Context128K
Q4_K_M7.5 GB
Roleplay
MI
Mistral 7B v0.2 hf
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 0.5b RBase
Qwen · QuantFactory
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
AH
Ahma 7B
Ahma · QuantFactory
Params7B
Context2K
Q4_K_M4.3 GB
GGUF
SM
Biggie SmoLlm 0.4B
SmolLM · QuantFactory
Params401M
Context2K
Q4_K_M0.3 GB
GGUF
GE
Gemma 2 2B ArliAI RPMax v1.1
Gemma · QuantFactory
Params2.6B
Context8K
Q4_K_M1.7 GB
GGUF
QW
Qwen1.5 MoE A2.7B
Qwen · QuantFactory
Params14.3B
Context8K
Q4_K_M9.5 GB
GGUF

Keep browsing