Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
ST
stablelm-2-zephyr-1_6b
Stablelm · second-state
Params1.64B
Context4K
Q4_K_M1.0 GB
GGUF
LL
Llama 3 Tulu 2 70b
Llama · RichardErkhov
Params70.6B
Context2K
Q4_K_M42.5 GB
GGUF
GR
ClinicalThought AI 8B
Granite · Raymond-dev-546730
Params8.17B
Context128K
Q4_K_M4.9 GB
ReasoningTool use
LL
Llama-3-Groq-8B-Tool-Use
Llama · second-state
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
GE
Gemma 2 2B Stheno Filtered
Gemma · QuantFactory
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
LL
Llama3 DocChat 1.0 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3 TTS 12Hz 0.6B CustomVoice
Qwen · BricksDisplay
Params652M
Context32K
Q4_K_M0.4 GB
GGUF
AN
Anima2
Anima · JusteLeo
Params2.09B
Context—
Q4_K_M1.2 GB
GGUF
LL
Llama 2 7b Pruned50 Retrained
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen1.5 32B Chat
Qwen · Qwen
Params32.5B
Context32K
Q4_K_M19.7 GB
GGUF
SE
Seed Coder 8B Instruct
Seed · second-state
Params8.25B
Context32K
Q4_K_M5.1 GB
Coding
KU
Kukedlc SuperMente 7B v4
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama 3 8B Instruct Finance RAG
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
CH
elinas_chronos-33b
Chronos · TheBloke
Params32.5B
Context2K
Q4_K_M19.6 GB
GGUF
QW
Qwen2.5-Omni-3B
Qwen · Mungert
Params3.4B
Context32K
Q4_K_M2.2 GB
GGUF
LL
OpenMath2 Llama3.1 8B
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
YI
Yi 34B GiftedConvo merged
Yi · TheBloke
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
LL
kykim0 Llama 2 7b ultrachat syn
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MY
Maxtopia 13B
Mythomax · QuantFactory
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
RE
reflect_llm8B_om2-mstlrg-300k460k-llm33-130k-t12_SFTt1-DPOsPSDPt1_b1
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
SA
SanGuoYanYi 6b
Sanguoyanyi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
QI
Qianfan-OCR
Qianfan · Mungert
Params4.42B
Context32K
Q4_K_M3.1 GB
ReasoningVision
UN
fblgit_una-cybertron-7b-v3-oma
Una · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IN
InternLM2
InternLM · QuantFactory
Params1.89B
Context8K
Q4_K_M1.2 GB
GGUF
LL
KISTI KONI KONI Llama3 8B Merged 20240724
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
PH
Phi 4 Mini Instruct
Phi · mmnga
Params3.84B
Context128K
Q4_K_M2.5 GB
Tool use
LL
read1337 Llama 3.1 8B bnb 4bit marketing
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
BA
T5 Base
Base · callgg
Params110M
Context512
Smallest0.1 GB
Coding
LI
lift
Lift · prithivMLmods
Params9.2B
Context256K
Q4_K_M5.8 GB
Vision

Other quants

Keep browsing