GGUF models by QuantFactory
1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.
SM
SmolLM 360M
SmolLM · QuantFactory
Params362M
Context2K
Q4_K_M0.3 GB
LL
Llama 3 8B Lexi Uncensored
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
BI
Bielik 11B v2.2 Instruct
Bielik · QuantFactory
Params11.2B
Context32K
Q4_K_M6.7 GB
GE
magnum v3 9b customgemma2
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
MI
Minerva 3B base RAG
Minerva · QuantFactory
Params2.89B
Context16K
Q4_K_M1.8 GB
MI
Hermes 2 Pro Mistral 7B
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
PE
PersianMind v1.0
Persianmind · QuantFactory
Params6.8B
Context2K
Q4_K_M4.1 GB
MI
mathstral 7B v0.1
Mistral · QuantFactory
Params7.25B
Context32K
Q4_K_M4.4 GB
LL
AMD Llama 135m
Llama · QuantFactory
Params134M
Context2K
Q4_K_M0.1 GB
OP
openchat 3.6 8b 20240522
Openchat · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
ST
stable code 3b
Stable · QuantFactory
Params2.8B
Context16K
Q4_K_M1.7 GB
LL
Meta Llama 3 70B Instruct
Llama · QuantFactory
Params70.6B
Context8K
Q4_K_M42.5 GB
BL
Blue Orchid 2x7b
Blue · QuantFactory
Params12.9B
Context8K
Q4_K_M7.8 GB
LL
CodeLlama 7B KStack
Llama · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
CO
Yi Coder 1.5B Chat
Coder · QuantFactory
Params1.48B
Context128K
Q4_K_M1.0 GB
LL
Llama3.1 8B Chinese Chat
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
MI
MiniCPM3 4B
MiniCPM · QuantFactory
Params4.07B
Context32K
Q4_K_M2.5 GB
LL
Llama 3.2 3B Agent007
Llama · QuantFactory
Params3.21B
Context128K
Q4_K_M2.0 GB
ST
starchat2 15b v0.1
Starchat · QuantFactory
Params16B
Context16K
Q4_K_M9.9 GB
LL
RoLlama2 7b Base
Llama · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
suzume llama 3 8B multilingual
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
LL
Meta Llama 3.1 8B Instruct abliterated
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
IN
InkubaLM 0.4B
Inkubalm · QuantFactory
Params664M
Context2K
Q4_K_M0.4 GB
MI
Mistral Nemo 12B ArliAI RPMax v1.2
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
LL
Hermes 3 Llama 3.1 8B lorablated
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
ST
starcoder2 15b
StarCoder · QuantFactory
Params16B
Context16K
Q4_K_M9.9 GB
LL
llm compiler 13b
Llm · QuantFactory
Params13B
Context16K
Q4_K_M7.9 GB
YI
Yi 1.5 9B Chat 16K
Yi · QuantFactory
Params8.83B
Context16K
Q4_K_M5.3 GB
WI
WizardLM 2 7B abliterated
Wizardlm · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
granite 7b instruct
Granite · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB