GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
UIGEN T3 8B Preview
Qwen · QuantFactory
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
LU
Lumimaid Magnum v4 12B
Lumimaid · QuantFactory
Params12.2B
Context128K
Q4_K_M7.5 GB
Tool use
LL
LLama 3.1 8B Matrix
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
PL
plamo 13b
Plamo · QuantFactory
Params13.1B
Context4K
Q4_K_M7.9 GB
GGUF
LL
Llama 3 SauerkrautLM 8b Instruct
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
VI
Vicuna 7b v1.5
Vicuna · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
SA
Sarashina2.1 1b
Sarashina · QuantFactory
Params1.41B
Context8K
Q4_K_M0.9 GB
GGUF
LL
dolphin 2.9 llama3 8b 256k
Llama · QuantFactory
Params8.03B
Context512K
Q4_K_M4.9 GB
GGUF
LL
llama 3.1 Asian Bllossom 8B Translator
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama 3 WhiteRabbitNeo 8B v2.0
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral v0.3 Instruct Matter Slim A
Mistral · QuantFactory
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
SA
Sailor2 L 8B Chat
Sailor · QuantFactory
Params8.55B
Context128K
Q4_K_M5.2 GB
GGUF
LL
Llama 3.2 Taiwan 3B
Llama · QuantFactory
Params3.61B
Context128K
Q4_K_M2.2 GB
Reasoning
LL
Llama3.1 DarkStorm Aspire 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Roleplay
LL
Llama 3 8B Magpie Pro SFT 100K v0.1
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MA
Fatgirl 8B
Magic · QuantFactory
Params8.41B
Context8K
Q4_K_M5.1 GB
GGUF
SI
SILMA 9B Instruct v1.0
Silma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
RO
Rocinante 12B V1A
Rocinante · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
QW
OpenThinker 7B
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
LL
DatacampLlama 3.1 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
CA
Captain_BMO 12B
Captain · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
LL
ArliAI Llama 3 8B Instruct DPO v0.2
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
gemma2 9b cpt sea lionv3 instruct
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
GE
BgGPT Gemma 2 9B IT v1.0
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
Mahou 1.5 mistral nemo 12B
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
NE
AceReason Nemotron 14B
Nemotron · QuantFactory
Params14.8B
Context128K
Q4_K_M9.0 GB
ReasoningTool use
LY
MN 12B Lyra v4
Lyra · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
LL
Ichigo llama3.1 s instruct v0.4
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
Minitron 4B Base
Minitron · QuantFactory
Params4.19B
Context4K
Q4_K_M2.7 GB
GGUF
LL
Llama 3 8B Tulu 330K
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Keep browsing