GGUF models by bartowski
1937 GGUF models quantized by bartowski on Hugging Face. Sizes run from 30M to 1030B parameters. The longest trained context is 10240K tokens.
LL
Llama 3.1 70B ArliAI RPMax v1.2
Llama · bartowski
Params70.6B
Context128K
Q4_K_M42.5 GB
LL
.\TroyDoesAI Llama 21B
Llama · bartowski
Params21.4B
Context32K
Q4_K_M12.9 GB
DE
DeepSeek-V2-Chat-0628
DeepSeek · bartowski
Params236B
Context160K
Q4_K_M142.5 GB
JA
Jan v1 Edge
Jan · bartowski
Params1.72B
Context40K
Q4_K_M1.1 GB
DS
Dsv2 Lite Coder_Codeio_Pp
Dsv · bartowski
Params15.7B
Context160K
Q4_K_M10.4 GB
FA
FastApply 1.5B v1.0
Fastapply · bartowski
Params1.54B
Context32K
Q4_K_M1.0 GB
PH
Phi 3 Context Obedient RAG
Phi · bartowski
Params3.82B
Context128K
Q4_K_M2.4 GB
GE
DiscoPOP-zephyr-7b-gemma
Gemma · bartowski
Params8.54B
Context8K
Q4_K_M6.3 GB
XL
xLAM 7b Fc R
Xlam · bartowski
Params6.91B
Context4K
Q4_K_M4.2 GB
MI
Hyperion 2.1 Mistral 7B
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
OR
OrpoLlama-3-8B
Orpollama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
gemma-1.1-2b-it
Gemma · bartowski
Params2.51B
Context8K
Q4_K_M1.6 GB
TA
TableLLM-13b
Tablellm · bartowski
Params13B
Context16K
Q4_K_M7.9 GB
LL
Meta-Llama-3-8B-Instruct-64k
Llama · bartowski
Params8.03B
Context64K
Q4_K_M4.9 GB
ST
starcoder2-15b-instruct
StarCoder · bartowski
Params16B
Context16K
Q4_K_M9.9 GB
LI
ozone research 2x lite
Lite · bartowski
Params14.8B
Context32K
Q4_K_M9.0 GB
WI
WizardLM-2-7B
Wizardlm · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
MA
mamba-2.8b-hf
Mamba · bartowski
Params2.77B
Context1024K
Q4_K_M2.0 GB
LL
AMD Llama 135m code
Llama · bartowski
Params134M
Context2K
Q4_K_M0.1 GB
MA
Math Genius 7B
Math · bartowski
Params7.25B
Context32K
Q4_K_M4.4 GB
OR
orca mini v8 0 70b
Orca · bartowski
Params70.6B
Context128K
Q4_K_M42.5 GB
RE
reader lm 0.5b
Reader · bartowski
Params494M
Context250K
Q4_K_M0.4 GB
GI
GigaChat3 702B A36B Preview
Gigachat · bartowski
Params702B
Context128K
Q4_K_M427.2 GB
FA
Fallen Command A 111B v1
Fallen · bartowski
Params111B
Context256K
Q4_K_M67.1 GB
YI
dolphin-2.9.1-yi-1.5-9b
Yi · bartowski
Params8.83B
Context12K
Q4_K_M5.3 GB
MI
Miniclaus qw1.5B UNAMGS
Miniclaus · bartowski
Params1.54B
Context128K
Q4_K_M1.0 GB
MI
MiS Firefly v0.1 22B
Mis · bartowski
Params22.2B
Context32K
Q4_K_M13.3 GB
GE
Tess-v2.5-Gemma-2-27B-alpha
Gemma · bartowski
Params27.2B
Context8K
Q4_K_M16.6 GB
LL
Pantheon-RP-1.0-8b-Llama-3
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
FA
Falcon3 7B Base
Falcon · bartowski
Params7.46B
Context32K
Q4_K_M4.6 GB