Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
VI
vicuna 7B v1.5
Vicuna · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen2.5 Coder 32B Instruct AWQ
Qwen · EasierAI
Params32.8B
Context128K
Q4_K_M19.9 GB
LL
hon9kon9ize Cantonese Llama 2 7B preview20240625
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 VL 8B Heretic Stable
Qwen · prithivMLmods
Params8.19B
Context256K
Q4_K_M5.0 GB
WA
Phantom Wan 14B FusionX
Wan · QuantStack
Params14.3B
Context—
Q4_K_M9.7 GB
OP
weyaxi_openhermes-2.5-neural-chat-7b-v3-1-7b
Openhermes · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LE
leolm_leo-hessianai-7b-chat
Leo · TheBloke
Params6.74B
Context8K
Q4_K_M4.1 GB
TE
migtissera_tess-m-v1.3
Tess · TheBloke
Params34.4B
Context195K
Q4_K_M20.7 GB
MI
Dolphin-Mistral-24B-Venice-Edition
Mistral · Mungert
Params23.6B
Context32K
Q4_K_M14.5 GB
LL
llm jp 3.1 13b instruct4
Llm · mmnga
Params13.7B
Context4K
Q4_K_M8.3 GB
CH
Chronomaid Storytelling 13B
Chronomaid · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
DE
triadparty_deepmoney-34b-200k-chat-evaluator
Deepmoney · TheBloke
Params34.4B
Context195K
Q4_K_M20.7 GB
PH
Phi 3 Mini 4k Instruct
Phi · SanctumAI
Params3.82B
Context4K
Q4_K_M2.4 GB
PU
PuddleJumper 13B V2
Puddlejumper · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
QW
Qwen2.5 14B
Qwen · mradermacher
Params14.8B
Context128K
Q4_K_M9.0 GB
LL
Llama 3.2 3B Instruct
Llama · QuantFactory
Params3.2B
Context128K
Q4_K_M2.0 GB
MY
MysticFusion 13B
Mysticfusion · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
NE
abideen_nexonimbus-7b
Nexonimbus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
YI
Yi 34B 200K DARE Merge v7
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
ST
L3 SthenoMaidBlackroot 8B V1
Sthenomaidblackroot · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
TH
TheSpice-7b-FT-v0.3.1
Thespice · duyntnet
Params7.24B
Context32K
Q4_K_M4.4 GB
FL
FLUX.1 Fill dev
Flux · second-state
Params11.9B
Context—
Smallest4.2 GB
LL
ShiningValiant 1.3
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
LL
PiSSA Llama 2 7b R128
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
SU
ariellee_superplatty-30b
Superplatty · TheBloke
Params32.5B
Context2K
Q4_K_M19.6 GB
OP
Openbuddy Yi1.5 34b v21.3 32k
Openbuddy · RichardErkhov
Params34.4B
Context128K
Q4_K_M20.7 GB
HE
HermesLimaRP L2 7B
Hermeslimarp · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
nondzu_mistral-7b-codealpaca-lora
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB