Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
HU
HunyuanVideo Avatar GGUF Experiment
Hunyuanvideo · lym00
Params13B
Context—
Q4_K_M8.0 GB
NO
notHumpback-Myx
Nothumpback · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
HA
Hathor Tahsin L3 8B v0.85
Hathor · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GI
GigaChat3-10B-A1.8B
Gigachat · Mungert
Params10.7B
Context256K
Q4_K_M6.7 GB
LL
scandukuri llama3 8b stargate m1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
MiniCPM5 2B
MiniCPM · XtrCom
Params2.52B
Context128K
Q4_K_M1.6 GB
ST
stable-code-instruct-3b
Stable · bartowski
Params2.8B
Context16K
Q4_K_M1.7 GB
GR
granite speech 4.0 1b
Granite · cstr
Params2.31B
Context—
Smallest2.9 GB
GE
Gemma 2b It
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
QW
Qwen2.5 7B Instruct
Qwen · ThomasBaruzier
Params7.62B
Context32K
Q4_K_M4.7 GB
DI
distilgpt2
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
LL
Llama 3 8B Web
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
NO
Nous-Capybara-3B-V1.9
Nous · duyntnet
Params2.8B
Context4K
Q4_K_M1.7 GB
OP
openchat-3.6-8b-20240522
Openchat · crusoeai
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
Mistral 7B Instruct DPO
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
LE
Leia Swallow 7b
Leia · RichardErkhov
Params6.83B
Context4K
Q4_K_M4.1 GB
MI
Devstral Small 2507
Mistral · gabriellarson
Params23.6B
Context128K
Q4_K_M14.3 GB
LL
Meta-Llama-3-13B-Instruct
Llama · LiteLLMs
Params13.9B
Context8K
Q4_K_M8.4 GB
SA
maritaca-ai_sabia-7b
Sabia · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GE
magnum v3 9b chatml
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
LL
mikael110_llama-2-70b-guanaco-qlora
Llama · TheBloke
Params69B
Context2K
Q4_K_M41.4 GB
GE
gemma-2b-it
Gemma · mlabonne
Params2.51B
Context8K
Q4_K_M1.5 GB
LL
Llama 3 8B Instruct 262k GGUF smashed
Llama · PrunaAI
Params8.03B
Context256K
Q4_K_M4.9 GB
LL
Llama 3.1 Swallow 8B Instruct v0.2
Llama · mmnga
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
uukuguy speechless codellama orca platypus 13b 0.10e
Llama · RichardErkhov
Params430M
Context—
Q4_K_M0.0 GB
MI
Dans PersonalityEngine V1.1.0 12b
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
TO
tomasonjo text2cypher demo 16bit
Tomasonjo · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
LL
Hermes-2-Pro-Llama-3-8B
Llama · duyntnet
Params8.03B
Context8K
Q4_K_M4.9 GB
BA
Baked Egglet 2B Model Stock
Baked · matrixportalx
Params2.61B
Context8K
Q4_K_M1.7 GB
LL
Taiwan LLaMa v1.0
Llama · audreyt
Params13B
Context2K
Q4_K_M7.9 GB