GGUF models by bartowski
1937 GGUF models quantized by bartowski on Hugging Face. Sizes run from 30M to 1030B parameters. The longest trained context is 10240K tokens.
ST
speechless-starcoder2-7b
StarCoder · bartowski
Params7.17B
Context16K
Q4_K_M4.4 GB
MI
kunoichi-lemon-royale-v2-32K-7B
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
IN
OREAL 32B
InternLM · bartowski
Params32.8B
Context128K
Q4_K_M19.9 GB
ST
StarDust 12b v2
Stardust · bartowski
Params12.2B
Context1000K
Q4_K_M7.5 GB
TR
Trinity Large TrueBase
Trinity · bartowski
Params399B
Context8K
Q4_K_M241.6 GB
HE
Hercules Stheno v1
Hercules · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
AR
Art v0 3B
Art · bartowski
Params3.09B
Context32K
Q4_K_M1.9 GB
PR
Nohobby L3.3 Prikol 70B v0.4
Prikol · bartowski
Params70.6B
Context128K
Q4_K_M42.5 GB
BE
Beeper King 22B
Beeper · bartowski
Params22.2B
Context128K
Q4_K_M13.3 GB
QW
Qwen2.5-1.5B-Open-R1-GRPO
Qwen · bartowski
Params1.54B
Context32K
Q4_K_M1.0 GB
QW
Chuluun Qwen2.5 72B v0.08
Qwen · bartowski
Params72.7B
Context128K
Q4_K_M47.4 GB
GE
Gemma 2 27b Chatml
Gemma · bartowski
Params27.2B
Context8K
Q4_K_M16.6 GB
LL
Scarlett-Llama-3-8B
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
SK
Skyfall 39B V1A WS
Skyfall · bartowski
Params39B
Context128K
Q4_K_M23.4 GB
QW
QwQ 32B Preview
Qwen · bartowski
Params32.8B
Context32K
Q4_K_M19.9 GB
LI
inclusionAI/Ling-1T
Ling · bartowski
Params1000B
Context32K
Q4_K_M608.4 GB
AT
Athene V2 Agent
Athene · bartowski
Params72.7B
Context32K
Q4_K_M47.4 GB
PA
Pantheon Rp 1.6 12b Nemo Kto
Pantheon · bartowski
Params12.2B
Context1000K
Q4_K_M7.5 GB
DA
Dans DangerousWinds V1.1.0 12b
Dans · bartowski
Params12.2B
Context128K
Q4_K_M7.5 GB
GE
Gemma 2b Aps It
Gemma · bartowski
Params2.51B
Context8K
Q4_K_M1.6 GB
MI
Q2.5 MS Mistoria 72b v2
Mistoria · bartowski
Params72.7B
Context32K
Q4_K_M47.4 GB
LL
llama 3 neural chat v1 8b
Llama · bartowski
Params8B
Context—
Q4_K_M4.9 GB
LI
ozone research 0x lite
Lite · bartowski
Params14.8B
Context32K
Q4_K_M9.0 GB
NE
L3.1 nemotron sunfall v0.7.0
Nemotron · bartowski
Params70.6B
Context128K
Q4_K_M42.5 GB
JE
Jedi 7B 1080p
Jedi · bartowski
Params7.62B
Context125K
Q4_K_M4.7 GB
DU
Duet Minitron8b v0.5
Duet · bartowski
Params8.41B
Context8K
Q4_K_M5.1 GB
LL
Llama3.1 8b_codeio_pp
Llama · bartowski
Params8.03B
Context128K
Q4_K_M4.9 GB
EN
Endurance 100B v1.1
Endurance · bartowski
Params100B
Context128K
Q4_K_M60.0 GB
BE
Behemoth ReduX 123B v1
Behemoth · bartowski
Params123B
Context128K
Q4_K_M73.2 GB
LL
Llama 3.1 Tulu 3 8B SFT
Llama · bartowski
Params8.03B
Context128K
Q4_K_M4.9 GB