GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
LLaMA 3 8B SFR SFT R
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
IN
instruction synthesizer
Instruction · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral Ita 7b
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CH
Chocolatine 3B Instruct DPO v1.2
Chocolatine · QuantFactory
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
TE
Teuken 7B instruct research v0.4
Teuken · QuantFactory
Params7.45B
Context4K
Q4_K_M5.0 GB
GGUF
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
Qwen2.5 7B
Qwen · QuantFactory
Params7.62B
Context128K
Q4_K_M4.7 GB
Tool use
GE
Leniachat Gemma 2b v0
Gemma · QuantFactory
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
ArliAI Llama 3 8B Cumulus v0.3.2
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
DE
deepseek math 7b instruct
DeepSeek · QuantFactory
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
MI
Infinity Instruct 7M Gen mistral 7B
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 2b Chinese it
Gemma · QuantFactory
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
LL
Llama3 8B Instruct Replete Adapted
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
CodingTool use
LL
Meta Llama 3 8B Instruct function calling json mode
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
YU
YugoGPT
Yugogpt · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NU
NuminaMath 7B TIR
Numinamath · QuantFactory
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
AY
aya 23 8B
Aya · QuantFactory
Params8B
Context4194304K
Q4_K_M5.1 GB
GGUF
GE
Gemma 2 9b It
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
UL
UltraMerge 7B
Ultramerge · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · QuantFactory
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
GE
Gemma 2 Ataraxy Doppel 9B
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
Llama3 ChatQA 1.5 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Replete LLM V2.5 Qwen 7b
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
OU
Oumuamua 7b instruct v2
Oumuamua · QuantFactory
Params7.33B
Context32K
Q4_K_M4.4 GB
GGUF
BI
BioMistral DARE NS
Biomistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BI
Bielik 11B v2
Bielik · QuantFactory
Params11.2B
Context32K
Q4_K_M6.7 GB
GGUF
QW
Qwen2.5 Coder 3B Instruct
Qwen · QuantFactory
Params3.4B
Context32K
Q4_K_M2.1 GB
CodingTool use
MI
Clinical BR Mistral 7B v0.2
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
L3 8B Stheno v3.3 32K
Stheno · QuantFactory
Params8.03B
Context32K
Q4_K_M4.9 GB
GGUF
AZ
MN Halide 12b v1.0
Azure · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Reasoning

Keep browsing