GGUF models by bartowski

1937 GGUF models quantized by bartowski on Hugging Face. Sizes run from 30M to 1030B parameters. The longest trained context is 10240K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
MadWizard-SFT-v2-Mistral-7b-v0.3
Mistral · bartowski
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
NX
Nxcode-CQ-7B-orpo
Nxcode · bartowski
Params7.25B
Context64K
Q4_K_M4.7 GB
Coding
RH
Rhea 72b v0.5 GGUF broken
Rhea · bartowski
Params72.3B
Context32K
Q4_K_M43.8 GB
GGUF
MA
MadWizardOrpoMistral-7b-v0.3
Madwizardorpomistral · bartowski
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Halu-8B-Llama3-v0.35
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GO
Goliath-longLORA-120b-rope8-32k-fp16
Goliath · bartowski
Params118B
Context4K
Q4_K_M70.6 GB
GGUF
FL
Flora 7B
Florabase · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QU
Quintellect-10.7B
Quintellect · bartowski
Params10.7B
Context32K
Q4_K_M6.5 GB
Coding
LL
nyun-llama3-62B
Llama · bartowski
Params62B
Context8K
Q4_K_M37.3 GB
GGUF
LL
Llama-3-70B-Synthia-v3.5
Llama · bartowski
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
LL
Higgs-Llama-3-70B
Llama · bartowski
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
MI
blossom v5 mistral 7b
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
DeepSeek V4 Flash
DeepSeek · bartowski
Params284B
Context1024K
Smallest156.0 GB
GGUF
MI
mistral 7b v0.1 layla v4 chatml
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Hercules 6.0 Llama 3.1 8B
Llama · bartowski
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
AI
airoboros-dpo-110b-3.3
Airoboros · bartowski
Params111B
Context32K
Q4_K_M67.2 GB
RoleplayTool use
CE
Celeste 12B V1.6
Celeste · bartowski
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
BL
blossom-v5-14b
Blossom · bartowski
Params14.2B
Context32K
Q4_K_M9.2 GB
GGUF
AR
Arco Put 6
Arco · bartowski
Params514M
Context8K
Q4_K_M0.3 GB
GGUF
AI
airoboros 34b 3.2
Airoboros · bartowski
Params34.4B
Context195K
Q4_K_M20.7 GB
RoleplayTool use
RH
L3 Rhaenys 8B
Rhaenys · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
Roleplay
TQ
Tqwendo 36b
Tqwendo · bartowski
Params35.7B
Context32K
Q4_K_M21.6 GB
CodingTool use
LL
airoboros-dpo-70b-3.3
Llama · bartowski
Params70.6B
Context8K
Q4_K_M42.5 GB
RoleplayTool use
CH
Chat2DB-SQL-7B
Chat · bartowski
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
BL
blossom v5 9b
Blossom · bartowski
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
LL
Llama-3-Orca-2.0-8B
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
AI
aixcoder-7b
Aixcoder · bartowski
Params7.43B
Context32K
Q4_K_M4.8 GB
Coding
KU
Beyonder-4x7B-v3
Kunoichi · bartowski
Params24.2B
Context32K
Q4_K_M14.6 GB
GGUF
LL
Llama-3-8B-Stroganoff
Llama · bartowski
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
YI
Yi 9B 200K
Yi · bartowski
Params8.83B
Context256K
Q4_K_M5.3 GB
GGUF

Keep browsing