GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama 3 8B Web
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral 7B Instruct DPO
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
magnum v3 9b chatml
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
Dans PersonalityEngine V1.1.0 12b
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
CodingRoleplay
LL
Meta Llama 3.1 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
BI
BioMistral MedMNX
Biomistral · QuantFactory
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
XL
xLAM 7b fc r
Xlam · QuantFactory
Params6.91B
Context4K
Q4_K_M4.2 GB
Tool use
LL
Llama 3 8B LexiFun Uncensored V1
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Roleplay
QW
Vikhr-Qwen-2.5-1.5B-Instruct
Qwen · QuantFactory
Params1.78B
Context32K
Q4_K_M1.1 GB
GGUF
LL
Infinity Instruct 7M Gen Llama3 1 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
IN
InternLM2
InternLM · QuantFactory
Params7.74B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen2 1.5B Ita
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
LL
Llama 3.1 8B Instuct Uz
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
DO
Dolus 14b Mini
Dolus · QuantFactory
Params11.5B
Context8K
Q4_K_M7.0 GB
Reasoning
LL
llama 3 nvidia ChatQA 1.5 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral NeMo Minitron 8B Instruct
Mistral · QuantFactory
Params8.41B
Context8K
Q4_K_M5.1 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
RoleplayTool use
DA
Daredevil 8B abliterated
Daredevil · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
YI
Yi 1.5 9B 32K
Yi · QuantFactory
Params8.83B
Context32K
Q4_K_M5.3 GB
GGUF
DE
Llama3.1 8B PRM Deepseek Data
DeepSeek · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MA
MAmmoTH2 8B
Mammoth · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Replete AI_Replete LLM Qwen2 7b_Beta Preview
Qwen · QuantFactory
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
LL
OpenBioLLM Llama3 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
math shepherd mistral 7b prm
Mistral · QuantFactory
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 9b It SimPO
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
Mistral 7B OpenOrca
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Reflection Llama 3.1 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
DE
DeepSeek R1 Distill Qwen 14B
DeepSeek · QuantFactory
Params14.8B
Context128K
Q4_K_M9.0 GB
ReasoningTool use
QW
CursorCore QW2.5 1.5B SR
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
TI
TinyLlama v1.1
Tinyllama · QuantFactory
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF

Keep browsing