Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
NE
Nethena 20B
Nethena · TheBloke
Params20B
Context4K
Q4_K_M12.0 GB
TI
tigerresearch_tigerbot-70b-chat-v4
Tigerbot · TheBloke
Params69.5B
Context2K
Q4_K_M41.8 GB
YI
Yi 1.5 6B Chat
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
CO
Code 13B
Code · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
AI
airoboros l2 13B 3.1
Airoboros · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
MI
Kant-Test-0.1-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PY
pythia 160m
Pythia · QuantFactory
Params162M
Context2K
Q4_K_M0.1 GB
OR
Orca2myth7.2
Orca · TheBloke
Params20B
Context4K
Q4_K_M12.0 GB
SQ
sqlcoder
Sqlcoder · TheBloke
Params15.8B
Context8K
Q4_K_M10.0 GB
ST
Starling LM 7B Beta
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LT
ltxv 13b 0.9.7 distilled
Ltx · ThePun1sher
Params13B
Context—
Q4_K_M8.8 GB
NI
Ninja v1 NSFW 128k
Ninja · mmnga
Params7.24B
Context128K
Q4_K_M4.4 GB
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
SW
Swallow 7B Instruct
Swallow · TheBloke
Params6.83B
Context4K
Q4_K_M4.1 GB
MI
microsoft_WizardLM-2-7B
Microsoft · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
imajev 2b
Qwen · mindchain
Params1.88B
Context256K
Q4_K_M1.3 GB
AI
aixsatoshi Honyaku 13b
Aixsatoshi · mmnga
Params13.1B
Context4K
Q4_K_M7.9 GB
PL
lilloukas_platypus-30b
Platypus · TheBloke
Params32.5B
Context2K
Q4_K_M19.6 GB
LL
Llama 3.2 3B Instruct Abliterated
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
WI
WizardLM 2 7B GGUF smashed
Wizardlm · PrunaAI
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral 7b Sandbagging Cb New
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen1.5 MoE A2.7B Chat
Qwen · duyntnet
Params14.3B
Context32K
Q4_K_M9.5 GB
ID
ideogram 4
Ideogram · molbal
Params9.28B
Context—
Smallest5.6 GB
CG
cgato TheSpice 7b FT ExperimentalOrca
Cgato · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
BO
Boogu Image Turbo GGUFs
Boogu · realrebelai
Params10.3B
Context—
Smallest6.8 GB
SY
migtissera_synthia-7b-v2.0
Synthia · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
openbuddy_openbuddy-mistral-7b-v13.1
Mistral · TheBloke
Params7.28B
Context32K
Q4_K_M4.4 GB
IN
Inkbot 13B 8k 0.2
Inkbot · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LL
Llama 3 8B ProLong 64k Base
Llama · RichardErkhov
Params8B
Context64K
Q4_K_M4.9 GB