Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen1.5-110B-Chat
Qwen · second-state
Params111B
Context32K
Q4_K_M67.2 GB
GGUF
QW
Qwen3.6-35B-A3B
Qwen · sphaela
Params35.5B
Context256K
Q4_K_M21.7 GB
Vision
LL
Llama-3-8B-Instruct-64k
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
ST
starcoder2-15b-instruct-v0.1
StarCoder · second-state
Params16B
Context16K
Q4_K_M9.9 GB
Coding
LL
CodeLlama 7B KStack
Llama · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
CO
Yi Coder 1.5B Chat
Coder · QuantFactory
Params1.48B
Context128K
Q4_K_M1.0 GB
Coding
MI
Mistral-7B-Instruct-v0.3
Mistral · SanctumAI
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen1.5-14B-Chat
Qwen · second-state
Params14.2B
Context32K
Q4_K_M9.2 GB
GGUF
MI
orangetin_fennec-mixtral-8x7b
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
Tool use
LL
Llama3.1 8B Chinese Chat
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
BA
ehartford_based-13b
Based · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
LL
codefuse-ai_codefuse-codellama-34b
Llama · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
Coding
OU
calderaai_13b-ouroboros
Ouroboros · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
ST
Stheno L2 13B
Stheno · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
JP
jpacifico Chocolatine 14B Instruct 4k DPO
Jpacifico · RichardErkhov
Params14B
Context4K
Q4_K_M8.6 GB
GGUF
PH
phi4
Phi · calcuis
Params14.7B
Context16K
Q4_K_M8.9 GB
GGUF
AG
agentlm 13B
Agentlm · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
MiniCPM3 4B
MiniCPM · QuantFactory
Params4.07B
Context32K
Q4_K_M2.5 GB
GGUF
ST
L3-8B-Stheno-v3.2
Stheno · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
SA
Sarashina2.2 0.5b Instruct v0.1
Sarashina · mmnga
Params793M
Context8K
Q4_K_M0.5 GB
GGUF
LL
Llama 3.2 3B Agent007
Llama · QuantFactory
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
TR
Trendyol LLM 7b chat v0.1
Trendyol · tolgadev
Params6.84B
Context4K
Q4_K_M4.2 GB
GGUF
LL
haqishen Llama 3 8B Japanese Instruct
Llama · mmnga
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
CodeQwen1.5-7B-Chat
Qwen · second-state
Params7.25B
Context64K
Q4_K_M4.7 GB
GGUF
ST
starchat2 15b v0.1
Starchat · QuantFactory
Params16B
Context16K
Q4_K_M9.9 GB
GGUF
SE
weyaxi_seraph-7b
Seraph · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
addis scribe streaming
Nemotron · addisai
Params99M
Context—
Smallest0.0 GB
GGUF
MI
openpipe_mistral-ft-optimized-1218
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ZE
zephyr-7b-sft-full-SPIN-iter3
Zephyr · duyntnet
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SQ
defog_sqlcoder-7b
Sqlcoder · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding

Other quants

Keep browsing