GGUF models by QuantFactory
1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.
GE
gemma 2 2b it Flight Multi Turn V3 DPO
Gemma · QuantFactory
Params2.61B
Context8K
Q4_K_M1.7 GB
LL
Vikhr Llama3.1 8B Instruct R 21 09 24
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GE
gemma 2 9b
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
LL
Llama 3.1 8B Stheno v3.4
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
ST
starcoder2 3b
StarCoder · QuantFactory
Params3.03B
Context16K
Q4_K_M1.9 GB
MA
MAmmoTH2 7B Plus
Mammoth · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
OL
OLMoE 1B 7B 0924
OLMo · QuantFactory
Params6.9B
Context4K
Q4_K_M4.2 GB
QW
Qwen2.5 0.5B
Qwen · QuantFactory
Params494M
Context32K
Q4_K_M0.4 GB
NO
Nous Hermes 2 SOLAR 10.7B
Nous · QuantFactory
Params10.7B
Context4K
Q4_K_M6.5 GB
MA
MagpieLM 4B Chat v0.1
Magpielm · QuantFactory
Params4.51B
Context128K
Q4_K_M2.8 GB
AH
Ahma 3B
Ahma · QuantFactory
Params3.63B
Context2K
Q4_K_M2.8 GB
PY
pythia 12b
Pythia · QuantFactory
Params11.8B
Context2K
Q4_K_M7.6 GB
MI
Hermes2 Gutenberg2 Mistral 7B
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral Nemo Base 2407
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
DE
DeepSeek Coder V2 Lite Instruct
DeepSeek · QuantFactory
Params15.7B
Context160K
Q4_K_M10.4 GB
MI
dolphin 2.1 mistral 7b
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 14B Gutenberg 1e Delta
Qwen · QuantFactory
Params14.8B
Context32K
Q4_K_M9.0 GB
SO
KULLM3
Solar · QuantFactory
Params10.7B
Context4K
Q4_K_M6.5 GB
QW
Qwen2.5 Sex
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
QW
Qwen2.5 Math 7B
Qwen · QuantFactory
Params7.6B
Context4K
Q4_K_M4.7 GB
HE
HelpingAI 9B
Helpingai · QuantFactory
Params8.83B
Context4K
Q4_K_M5.3 GB
LL
Dorna Llama3 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
MI
Codestral 22B v0.1
Mistral · QuantFactory
Params22.2B
Context32K
Q4_K_M13.3 GB
VI
MN Violet Lotus 12B
Violet · QuantFactory
Params12.2B
Context128K
Q4_K_M7.5 GB
MI
Mistral 7B v0.2 hf
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 0.5b RBase
Qwen · QuantFactory
Params494M
Context32K
Q4_K_M0.4 GB
AH
Ahma 7B
Ahma · QuantFactory
Params7B
Context2K
Q4_K_M4.3 GB
SM
Biggie SmoLlm 0.4B
SmolLM · QuantFactory
Params401M
Context2K
Q4_K_M0.3 GB
GE
Gemma 2 2B ArliAI RPMax v1.1
Gemma · QuantFactory
Params2.6B
Context8K
Q4_K_M1.7 GB
QW
Qwen1.5 MoE A2.7B
Qwen · QuantFactory
Params14.3B
Context8K
Q4_K_M9.5 GB