Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
rana shahroz mistral 7b roleplay alpaca merge
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
QW
Qwen2 VL 2B Instruct
Qwen · second-state
Params1.54B
Context32K
Q4_K_M1.0 GB
Vision
PH
migtissera Tess v2.5 Phi 3 medium 128k 14B
Phi · RichardErkhov
Params14B
Context128K
Q4_K_M8.6 GB
GGUF
QW
Qwen3 Coder 30B A3B Instruct
Qwen · second-state
Params30.5B
Context256K
Q4_K_M18.6 GB
Coding
DA
DaringMaid 20B
Daringmaid · TheBloke
Params20B
Context4K
Q4_K_M12.0 GB
GGUF
AU
Augmental Unholy 13B
Augmental · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
ausboss_llama-30b-supercot
Llama · TheBloke
Params32.5B
Context2K
Q4_K_M19.6 GB
GGUF
DA
pocketdoc_dans-totsirocco-7b
Dans · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LU
LUNA SOLARkrautLM Instruct
Luna · TheBloke
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
WA
wan 1.3b
Wan · bykhan
Params127M
Context—
Q4_K_M1.0 GB
GGUF
QW
dolphin 2.9.2 qwen2 7b
Qwen · QuantFactory
Params7.62B
Context128K
Q4_K_M4.7 GB
CodingTool use
MI
MistralLite 7B
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
weyaxi_openhermes-2.5-neural-chat-7b-v3-2-7b
Openhermes · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
JA
japanese stablelm base beta 70B
Japanese · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
LL
ELYZA-japanese-Llama-2-7b-fast-instruct
Llama · mmnga
Params6.85B
Context4K
Q4_K_M4.2 GB
GGUF
ST
starcoder2-15b
StarCoder · QuantFactory
Params16B
Context16K
Q4_K_M9.9 GB
Coding
MI
MiniMax H3
Minimax · ldov
Params20.1B
Context—
Q4_K_M18.2 GB
GGUF
FL
FLOR 1.3B Instructed
Flor · xaviviro
Params1.41B
Context4K
Q4_K_M0.9 GB
GGUF
HO
K2 Horizon 0.9B
Horizon · NANI-Nithin
Params1.08B
Context128K
Q4_K_M0.7 GB
GGUF
NE
mlabonne_neuralpipe-7b-slerp
Neuralpipe · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SY
maywell_synatra-v0.1-7b-instruct
Synatra · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TO
tokyotech llm Swallow 7b instruct hf
Tokyotech · RichardErkhov
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
LL
CodeLlama 7b Instruct hf
Llama · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
TH
The Trinity Coder 7B
The · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
MI
nickynicky_mistral-7b-openorca-oasst_top1_2023-08-25-v2
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LE
LevelField
Levelfield · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
LL
Turkish Llama 8b Instruct v0.1
Llama · ytu-ce-cosmos
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3 8B Instruct Gradient 1048k GGUF smashed
Llama · PrunaAI
Params8B
Context1024K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 Coder 1.5B
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
YI
Yi 34B
Yi · TheBloke
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF

Other quants

Keep browsing