GGUF models by bartowski
1937 GGUF models quantized by bartowski on Hugging Face. Sizes run from 30M to 1030B parameters. The longest trained context is 10240K tokens.
MI
NeuralHyperion 2.0 Mistral 7B
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama-3-8B-Instruct-262k
Llama · bartowski
Params8.03B
Context256K
Q4_K_M4.9 GB
DA
h2o-danube2-1.8b-chat
Danube · bartowski
Params1.83B
Context8K
Q4_K_M1.1 GB
OP
OpenCrystal 15B L3 v2
Opencrystal · bartowski
Params15B
Context8K
Q4_K_M9.1 GB
LL
maverick-llama3-8B
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
ST
Starling-LM-10.7B-beta
Starling · bartowski
Params10.7B
Context8K
Q4_K_M6.5 GB
LL
Llama-3-Perky-Pat-Instruct-8B
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
BE
Behemoth 123B v1.2
Behemoth · bartowski
Params123B
Context128K
Q4_K_M73.2 GB
LL
Halu 8B Llama3 Blackroot
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
WE
Starling Monarch Westlake Garten 7B v0.1
Westlake · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
MO
Monstral 123B
Monstral · bartowski
Params123B
Context128K
Q4_K_M73.2 GB
JA
AI21 Jamba Large 1.7
Jamba · bartowski
Params399B
Context256K
Q4_K_M240.4 GB
LL
llama-3-meerkat-8b-v1.0
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Llama-3-Alpha-Centauri-v0.1
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
TU
Tunguska 39B V1A Workspace
Tunguska · bartowski
Params39B
Context128K
Q4_K_M23.4 GB
BE
Behemoth 123B v1.1
Behemoth · bartowski
Params123B
Context128K
Q4_K_M73.2 GB
LL
Llama-3-8B-Instruct-MopeyMule
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
HA
Hathor_Stable-L3-8B-v0.5
Hathor · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
ME
MetaStone L1 7B
Metastone · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
AG
Agatha 111B v1
Agatha · bartowski
Params111B
Context256K
Q4_K_M67.1 GB
LL
New-Dawn-Llama-3-70B-32K-v1.0
Llama · bartowski
Params70.6B
Context32K
Q4_K_M42.5 GB
MI
Tess-2.0-Mixtral-v0.2
Mistral · bartowski
Params46.7B
Context32K
Q4_K_M28.4 GB
AR
arcee lite
Arcee · bartowski
Params1.78B
Context32K
Q4_K_M1.1 GB
RE
Retrograde Omega M 22B v1.0
Retrograde · bartowski
Params22.2B
Context128K
Q4_K_M13.3 GB
ST
Starling-LM-7B-beta-ExPO
Starling · bartowski
Params7.24B
Context8K
Q4_K_M4.4 GB
NE
Rombos LLM V2.6 Nemotron 70b
Nemotron · bartowski
Params70.6B
Context128K
Q4_K_M42.5 GB
GE
Replete LLM gemma 9b test merged 250k
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
LL
llama3-turbcat-instruct-8b
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
Hyperion 3.0 Mistral 7B DPO
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
YI
Faro-Yi-9B-DPO
Yi · bartowski
Params8.83B
Context32K
Q4_K_M5.3 GB