Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
MA
openaccess-ai-collective_manticore-13b-chat-pyg
Manticore · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
AU
AuraFlow v0.3
Auraflow · city96
Params6.86B
Context—
Q4_K_M4.0 GB
QW
MiroThinker-v1.5-30B
Qwen · Mungert
Params30.5B
Context256K
Q4_K_M18.9 GB
KA
KafkaLM 70B German V0.1
Kafkalm · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
MI
cognitivecomputations_dolphin-2.6-mistral-7b-dpo-laser
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LE
Lemur 70B Chat v1
Lemur · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
LL
togethercomputer_llama-2-7b-32k-instruct
Llama · TheBloke
Params6.74B
Context32K
Q4_K_M4.1 GB
ME
q-bert_metamath-cybertron-starling
Metamath · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
SM
SmolLM 135M
SmolLM · QuantFactory
Params135M
Context2K
Q4_K_M0.1 GB
WI
ehartford_wizard-vicuna-30b-uncensored
Wizard · TheBloke
Params32.5B
Context2K
Q4_K_M19.6 GB
GR
Granite 8b Code Base 128k
Granite · RichardErkhov
Params8.05B
Context125K
Q4_K_M4.9 GB
WA
Wan Move 14B 480P
Wan · vantagewithai
Params16.4B
Context—
Q4_K_M11.3 GB
LL
openassistant_llama2-13b-orca-8k-3319
Llama · TheBloke
Params13B
Context8K
Q4_K_M7.9 GB
LL
ehartford_samantha-1.11-codellama-34b
Llama · TheBloke
Params33.7B
Context2K
Q4_K_M20.2 GB
BE
Bernini R GGUF ComfyUI
Bernini · vantagewithai
Params14.3B
Context—
Q4_K_M9.7 GB
JA
japanese stablelm instruct beta 70B
Japanese · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
YI
01 ai Yi 6B
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
CL
clef flash
Clef · NANI-Nithin
Params8.95B
Context256K
Q4_K_M5.6 GB
YI
01 ai Yi 34B Chat
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
PY
Python Code 13B
Python · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LL
Meta Llama 3.1 8B Instruct
Llama · SanctumAI
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen2.5 14B Instruct
Qwen · ThomasBaruzier
Params14.8B
Context32K
Q4_K_M9.0 GB
GR
Granite 3b Code Instruct
Granite · RichardErkhov
Params3.48B
Context2K
Q4_K_M2.1 GB
HU
HunyuanVideo I2V
Hunyuanvideo · city96
Params12.8B
Context—
Q4_K_M7.9 GB
LL
Llama 2 Coder 7B
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GL
AutoGLM Phone 9B
GLM · Luckybalabala
Params9.4B
Context64K
Q4_K_M6.2 GB
DA
DataPilot ArrowPro 7B KUJIRA
Datapilot · mmnga
Params7.24B
Context4K
Q4_K_M4.4 GB
IM
Z Image Turbo
Image · vantagewithai
Params6.15B
Context—
Q4_K_M5.0 GB
MI
E5 Mistral 7B Instruct Embedding
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
dolphin 2.9 llama3 8b
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB