GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
SM
SmolLM 360M
SmolLM · QuantFactory
Params362M
Context2K
Q4_K_M0.3 GB
Tool use
LL
Llama 3 8B Lexi Uncensored
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
BI
Bielik 11B v2.2 Instruct
Bielik · QuantFactory
Params11.2B
Context32K
Q4_K_M6.7 GB
GGUF
GE
magnum v3 9b customgemma2
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
Minerva 3B base RAG
Minerva · QuantFactory
Params2.89B
Context16K
Q4_K_M1.8 GB
GGUF
MI
Hermes 2 Pro Mistral 7B
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
PE
PersianMind v1.0
Persianmind · QuantFactory
Params6.8B
Context2K
Q4_K_M4.1 GB
GGUF
MI
mathstral 7B v0.1
Mistral · QuantFactory
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
LL
AMD Llama 135m
Llama · QuantFactory
Params134M
Context2K
Q4_K_M0.1 GB
GGUF
OP
openchat 3.6 8b 20240522
Openchat · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
ST
stable code 3b
Stable · QuantFactory
Params2.8B
Context16K
Q4_K_M1.7 GB
Coding
LL
Meta Llama 3 70B Instruct
Llama · QuantFactory
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
BL
Blue Orchid 2x7b
Blue · QuantFactory
Params12.9B
Context8K
Q4_K_M7.8 GB
GGUF
LL
CodeLlama 7B KStack
Llama · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
CO
Yi Coder 1.5B Chat
Coder · QuantFactory
Params1.48B
Context128K
Q4_K_M1.0 GB
Coding
LL
Llama3.1 8B Chinese Chat
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MI
MiniCPM3 4B
MiniCPM · QuantFactory
Params4.07B
Context32K
Q4_K_M2.5 GB
GGUF
LL
Llama 3.2 3B Agent007
Llama · QuantFactory
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
ST
starchat2 15b v0.1
Starchat · QuantFactory
Params16B
Context16K
Q4_K_M9.9 GB
GGUF
LL
RoLlama2 7b Base
Llama · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
suzume llama 3 8B multilingual
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Meta Llama 3.1 8B Instruct abliterated
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
IN
InkubaLM 0.4B
Inkubalm · QuantFactory
Params664M
Context2K
Q4_K_M0.4 GB
GGUF
MI
Mistral Nemo 12B ArliAI RPMax v1.2
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
Hermes 3 Llama 3.1 8B lorablated
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
Tool use
ST
starcoder2 15b
StarCoder · QuantFactory
Params16B
Context16K
Q4_K_M9.9 GB
Coding
LL
llm compiler 13b
Llm · QuantFactory
Params13B
Context16K
Q4_K_M7.9 GB
GGUF
YI
Yi 1.5 9B Chat 16K
Yi · QuantFactory
Params8.83B
Context16K
Q4_K_M5.3 GB
GGUF
WI
WizardLM 2 7B abliterated
Wizardlm · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
granite 7b instruct
Granite · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF

Keep browsing