GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
OR
Orca 2 13b
Orca · QuantFactory
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
GR
granite 7b base
Granite · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
PH
Phi 3.5 mini ITA
Phi · QuantFactory
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
medicine Llama3 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3.1 8B Instruct Abliterated_Via_Adapter
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama Guard 3 1B
Llama · QuantFactory
Params1.5B
Context128K
Q4_K_M1.0 GB
GGUF
PR
prem 1B SQL
Prem · QuantFactory
Params1.35B
Context16K
Q4_K_M0.9 GB
GGUF
LL
Medical Llama3 v2
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
llama 161M 100B
Llama · QuantFactory
Params162M
Context1K
Q4_K_M0.1 GB
GGUF
LY
Lyra4 Gutenberg2 12B
Lyra · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
MA
Marco o1
Marco · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
GGUF
GE
gemma 2 baku 2b it
Gemma · QuantFactory
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
BI
BioMistral 7B
Biomistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LI
OmniLing V1 8b experimental
Lite · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
RE
Refact 1 6 base
Refact · QuantFactory
Params1.59B
Context4K
Q4_K_M1.0 GB
Coding
SM
SmolLM 1.7B
SmolLM · QuantFactory
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
GE
Gemma2 9B TitanFusion
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
TE
TextBase 7B v0.1
Textbase · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
ScaduTorrent1.1 8b model stock
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Roleplay
LL
llama 3 Korean Bllossom 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
NI
L3.1 Celestial Stone 2x8B
Niitama · QuantFactory
Params13.7B
Context128K
Q4_K_M8.3 GB
GGUF
DE
DeepSeek R1 Distill Qwen 1.5B
DeepSeek · QuantFactory
Params1.78B
Context128K
Q4_K_M1.1 GB
Reasoning
LL
Llama 3 8b 64k PoSE
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Unichat llama3 Chinese 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PH
calme-2.1-phi3.5-4b
Phi · QuantFactory
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
ZE
zephyr 7b alpha
Zephyr · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
WA
wavecoder ultra 6.7b
Wavecoder · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
OL
OLMoE 1B 7B 0924 Instruct
OLMo · QuantFactory
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
GE
Gemma 2 9b It WPO HB
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
QW
lb reranker 0.5B v1.0
Qwen · QuantFactory
Params630M
Context32K
Q4_K_M0.5 GB
EmbeddingTool use

Keep browsing