Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
teknium_hermes-trismegistus-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MI
Chinese Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.5B
Context32K
Q4_K_M4.5 GB
GGUF
MI
fpham_generate_question_mistral_7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NA
calderaai_naberius-7b
Naberius · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Llama 3 Patronus Lynx 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
HO
HornyEchidna 13B v0.1
Hornyechidna · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
SO
SOLAR Platypus 10.7B v2
Solar · TheBloke
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
AU
Augmental ReMM 13B
Augmental · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
GP
gpt2
Gpt · QuantFactory
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
CH
chujiezheng Starling LM 7B beta ExPO
Chujiezheng · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
QW
Qwen2 1.5B Instruct
Qwen · Goekdeniz-Guelmez
Params1.78B
Context32K
Q4_K_M1.1 GB
GGUF
LL
Llama-3-8B-Instruct-Gradient-1048k
Llama · crusoeai
Params8B
Context1024K
Q4_K_M4.9 GB
GGUF
MI
uukuguy_speechless-code-mistral-7b-v1.0
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
MA
openaccess-ai-collective_manticore-13b
Manticore · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
RH
rho-math-1b-interpreter-v0.1
Rho · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
PI
maywell_pivot-sus-rp
Pivot · TheBloke
Params34.4B
Context8K
Q4_K_M20.7 GB
Roleplay
QW
Qwen2.5 1.5B Instruct
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
ZI
idea-ccnl_ziya-coding-34b-v1.0
Ziya · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
GGUF
CA
canary 1b v2
Canary · cstr
Params980M
Context—
Smallest0.0 GB
GGUF
QW
Qwen2 Audio 7B
Qwen · NexaAI
Params7.75B
Context8K
Q4_K_M4.8 GB
GGUF
MI
UCLA AGI Mistral7B PairRM SPPO Iter3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AI
bhenrym14_airoboros-l2-13b-2.1-yarn-64k
Airoboros · TheBloke
Params13B
Context64K
Q4_K_M7.9 GB
GGUF
EH
ehartford_wizardlm-33b-v1.0-uncensored
Ehartford · TheBloke
Params32.5B
Context2K
Q4_K_M19.6 GB
GGUF
CH
Chronos 70B v2
Chronos · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
Roleplay
PR
T pro it 2.1
Pro · t-tech
Params32.8B
Context40K
Q4_K_M19.8 GB
GGUF
MI
MistralMakise Merged 13B
Mistral · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
AU
Augmental 13B v1.50 A
Augmental · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
DE
deepseek coder 1.3B kexer
DeepSeek · QuantFactory
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
CO
beowolx_codeninja-1.0-openchat-7b
Codeninja · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
QW
Qwen2.5 32B Instruct Abliterated
Qwen · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB
GGUF

Other quants

Keep browsing