Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
OpenBioLLM Llama3 70B GGUF smashed
Llama · PrunaAI
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
MI
MiniMax H3 Hybrid b25 49
Minimax · hoidhxd
Params20.1B
Context—
Smallest11.4 GB
GGUF
LL
Llama 2 13b Hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
RW
rwkv 6 finch 1b6
Rwkv · latestissue
Params1.6B
Context1024K
Smallest0.7 GB
GGUF
LL
OrionStar Yi 34B Chat Llama
Llama · second-state
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
TI
jeff31415_tinyllama-1.1b-1t-openorca
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
PH
phi 2 orange
Phi · TheBloke
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
YI
taozi555_rpbird-yi-34b-200k
Yi · TheBloke
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
LL
Llama3 8b Cpt Sea Lionv2 Instruct
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
UC
UCLA AGI zephyr 7b sft full SPIN iter3
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
YO
Youtu-VL-4B-Instruct
Youtu · Mungert
Params4.9B
Context32K
Q4_K_M3.1 GB
Vision
QW
QwQ 32B Preview
Qwen · ThomasBaruzier
Params32.8B
Context32K
Q4_K_M19.9 GB
Reasoning
DE
realtreetune deepseekmath 7b sft GSM8K
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
FR
Free Sydney V2 13B
Free · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
GR
Granite 34b Code Base 8k
Granite · RichardErkhov
Params33.7B
Context8K
Q4_K_M21.4 GB
Coding
MI
K2S3 Mistral 7b v1.46
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
EV
EverythingLM 13b V2 16K
Everythinglm · TheBloke
Params13B
Context—
Q4_K_M6.3 GB
GGUF
SY
migtissera_synthia-7b-v3.0
Synthia · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
intel_neural-chat-7b-v3-1
Neural · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Dolphin Llama2 7B
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
SO
SOLARC MOE 10.7Bx4
Solarc · TheBloke
Params36B
Context4K
Q4_K_M20.4 GB
GGUF
MY
MythoLogic L2 13B
Mythologic · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
WA
Wan2.1 Fun 14B Control
Wan · city96
Params16.4B
Context—
Q4_K_M10.9 GB
GGUF
GR
granite 4.2 3b
Granite · NANI-Nithin
Params3.66B
Context128K
Q4_K_M2.2 GB
ReasoningTool use
QW
Qwen2.5 0.5B
Qwen · mradermacher
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
QW
Qwen2.5 0.5B Instruct
Qwen · second-state
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
KU
Kunoichi DPO v2 7B
Kunoichi · brittlewis12
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 Reranker 4B
Qwen · giladgd
Params4B
Context40K
Q4_K_M2.5 GB
Embedding
LL
Llama 3.2 3B Agent007 Coder
Llama · QuantFactory
Params3.21B
Context128K
Q4_K_M2.0 GB
Coding
PY
Pygmalion 2 13B SuperCOT
Pygmalion · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF

Other quants

Keep browsing