GGUF models by bartowski
1937 GGUF models quantized by bartowski on Hugging Face. Sizes run from 30M to 1030B parameters. The longest trained context is 10240K tokens.
MI
MadWizard-SFT-v2-Mistral-7b-v0.3
Mistral · bartowski
Params7.25B
Context32K
Q4_K_M4.4 GB
NX
Nxcode-CQ-7B-orpo
Nxcode · bartowski
Params7.25B
Context64K
Q4_K_M4.7 GB
RH
Rhea 72b v0.5 GGUF broken
Rhea · bartowski
Params72.3B
Context32K
Q4_K_M43.8 GB
MA
MadWizardOrpoMistral-7b-v0.3
Madwizardorpomistral · bartowski
Params7.25B
Context32K
Q4_K_M4.4 GB
LL
Halu-8B-Llama3-v0.35
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
GO
Goliath-longLORA-120b-rope8-32k-fp16
Goliath · bartowski
Params118B
Context4K
Q4_K_M70.6 GB
FL
Flora 7B
Florabase · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
QU
Quintellect-10.7B
Quintellect · bartowski
Params10.7B
Context32K
Q4_K_M6.5 GB
LL
nyun-llama3-62B
Llama · bartowski
Params62B
Context8K
Q4_K_M37.3 GB
LL
Llama-3-70B-Synthia-v3.5
Llama · bartowski
Params70.6B
Context8K
Q4_K_M42.5 GB
LL
Higgs-Llama-3-70B
Llama · bartowski
Params70.6B
Context8K
Q4_K_M42.5 GB
MI
blossom v5 mistral 7b
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
DeepSeek V4 Flash
DeepSeek · bartowski
Params284B
Context1024K
Smallest156.0 GB
MI
mistral 7b v0.1 layla v4 chatml
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Hercules 6.0 Llama 3.1 8B
Llama · bartowski
Params8.03B
Context128K
Q4_K_M4.9 GB
AI
airoboros-dpo-110b-3.3
Airoboros · bartowski
Params111B
Context32K
Q4_K_M67.2 GB
CE
Celeste 12B V1.6
Celeste · bartowski
Params12.2B
Context1000K
Q4_K_M7.5 GB
BL
blossom-v5-14b
Blossom · bartowski
Params14.2B
Context32K
Q4_K_M9.2 GB
AR
Arco Put 6
Arco · bartowski
Params514M
Context8K
Q4_K_M0.3 GB
AI
airoboros 34b 3.2
Airoboros · bartowski
Params34.4B
Context195K
Q4_K_M20.7 GB
RH
L3 Rhaenys 8B
Rhaenys · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
TQ
Tqwendo 36b
Tqwendo · bartowski
Params35.7B
Context32K
Q4_K_M21.6 GB
LL
airoboros-dpo-70b-3.3
Llama · bartowski
Params70.6B
Context8K
Q4_K_M42.5 GB
CH
Chat2DB-SQL-7B
Chat · bartowski
Params6.74B
Context16K
Q4_K_M4.1 GB
BL
blossom v5 9b
Blossom · bartowski
Params8.83B
Context4K
Q4_K_M5.3 GB
LL
Llama-3-Orca-2.0-8B
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
AI
aixcoder-7b
Aixcoder · bartowski
Params7.43B
Context32K
Q4_K_M4.8 GB
KU
Beyonder-4x7B-v3
Kunoichi · bartowski
Params24.2B
Context32K
Q4_K_M14.6 GB
LL
Llama-3-8B-Stroganoff
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
YI
Yi 9B 200K
Yi · bartowski
Params8.83B
Context256K
Q4_K_M5.3 GB