GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
HTML Pruner Llama 1B
Llama · QuantFactory
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
Meta Llama 3.1 8B Text to SQL
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
Gemmasutra Mini 2B v1
Gemma · QuantFactory
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GE
gemma 2 2b
Gemma · QuantFactory
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GR
granite 3.0 2b base
Granite · QuantFactory
Params2.53B
Context4K
Q4_K_M1.5 GB
GGUF
LL
ArliAI Llama 3 8B Dolfin v0.6 Abliterated
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
DU
Duet Minitron8b v0.5
Duet · QuantFactory
Params8.41B
Context8K
Q4_K_M5.1 GB
GGUF
TI
TinyMistral 248M v3
Tinymistral · QuantFactory
Params248M
Context32K
Q4_K_M0.2 GB
GGUF
QW
Qwen 2.5 3b Evol CoT
Qwen · QuantFactory
Params3.09B
Context32K
Q4_K_M1.9 GB
RoleplayTool use
QW
cybertron v4 qw7B MGS
Qwen · QuantFactory
Params7.62B
Context128K
Q4_K_M4.7 GB
Tool use
QW
OpenThinker2 7B
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
LL
Llama 3 8B WizardLM 196K
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SM
SmolLM 1.7B Instruct
SmolLM · QuantFactory
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
VI
Violet Twilight v0.2
Violet · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Roleplay
LL
SOVL Llama3 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3.1 SuperNova 8B Lite TIES with Base
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 medium 4k instruct
Phi · QuantFactory
Params14B
Context4K
Q4_K_M8.6 GB
Coding
YI
Yi 1.5 9B Chat GGUF v2
Yi · QuantFactory
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
LL
Llama 3.1 EIRAI 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
RA
Rabat chatbot
Rabat · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
NovaSpark
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Roleplay
GR
granite 3.0 8b base
Granite · QuantFactory
Params8.17B
Context4K
Q4_K_M4.9 GB
GGUF
GE
wiroai turkish llm 9b
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
GE
gemma 3 270m
Gemma · QuantFactory
Params436M
Context32K
Q4_K_M0.4 GB
GGUF
NE
Nemomix v1.0 12B
Nemomix · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
VI
st vicuna v1.3 5.5b ppl
Vicuna · QuantFactory
Params5.52B
Context2K
Q4_K_M3.3 GB
GGUF
LL
meditron 7b
Llama · QuantFactory
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
SA
Sarvam 1
Sarvam · QuantFactory
Params2.53B
Context8K
Q4_K_M1.5 GB
Tool use
MI
Chronos Gold 12B 1.0
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Roleplay
LL
Llama 3.1 WhiteRabbitNeo 2 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF

Keep browsing