GGUF models by QuantFactory

1418 GGUF models quantized by QuantFactory on Hugging Face. Sizes run from 9M to 225B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
GE
EZO Humanities 9B gemma 2 it
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
Llama 3 8B Magpie Pro SFT 200K v0.1
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
VI
st vicuna v1.3 10.5b ppl
Vicuna · QuantFactory
Params10.5B
Context2K
Q4_K_M6.3 GB
GGUF
SU
L3.1 8B sunfall stheno v0.6.1
Sunfall · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Lite Mistral 150M v2 Instruct
Mistral · QuantFactory
Params157M
Context2K
Q4_K_M0.1 GB
GGUF
NI
L3 Rhaenys 8B
Niitama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Roleplay
AI
aixcoder 7b base
Aixcoder · QuantFactory
Params7B
Context32K
Q4_K_M4.8 GB
Coding
DA
h2o danube3 4b base
Danube · QuantFactory
Params3.96B
Context8K
Q4_K_M2.4 GB
GGUF
SM
Smegmma Deluxe 9B v1
Smegmma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
SA
SauerkrautLM UNA SOLAR Instruct
Sauerkrautlm · QuantFactory
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
QW
OlympicCoder 7B
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingReasoningTool use
LL
vietnamese llama2 7b 40GB
Llama · QuantFactory
Params6.86B
Context4K
Q4_K_M4.2 GB
GGUF
AR
Arcee Spark
Arcee · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Fireball Meta Llama 3.1 8B Instruct Agent 0.003 128K
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
ST
L3 Nymeria Maid 8B
Stheno · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Roleplay
HO
Honyaku Multi Translator Swallow ms7b
Honyaku · QuantFactory
Params7.33B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Replete LLM gemma 9b test merged 250k
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LU
Lumimaid v0.2 8B
Lumimaid · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LI
Lite Oute 1 65M Instruct
Lite · QuantFactory
Params65M
Context2K
Q4_K_M0.0 GB
GGUF
TI
TinyMistral 248M v2.5 Instruct
Tinymistral · QuantFactory
Params248M
Context32K
Q4_K_M0.2 GB
Coding
LL
llama 3 fantasy writer 8b
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral Nemo Gutenberg Doppel 12B
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
GE
Gemma Dental it 07 merged
Gemma · QuantFactory
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
Stella mistral nemo 12B
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
Llama3 8b SVA FT final
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
IR
Irbis 7b v0.1
Irbis · QuantFactory
Params7.47B
Context32K
Q4_K_M4.5 GB
GGUF
LL
Llama 3 Instruct 8B SimPO
Llama · QuantFactory
Params8.03B
Context8K
Smallest5.7 GB
GGUF
SO
SOLAR 10.7B Instruct v1.0 uncensored
Solar · QuantFactory
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
HA
Hathor Fractionate L3 8B v.05
Hathor · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
TA
L3 8B Tamamo v1
Tamamo · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Keep browsing