Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
ZhangShenao SELM Llama 3 8B Instruct iter 1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Dolphin3.0 Llama3.1 1B abliterated
Llama · NexesQuants
Params1.24B
Context128K
Smallest0.9 GB
GGUF
LL
Meta Llama 3.1 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · SanctumAI
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
BI
BioMistral MedMNX
Biomistral · QuantFactory
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
XL
xLAM 7b fc r
Xlam · QuantFactory
Params6.91B
Context4K
Q4_K_M4.2 GB
Tool use
LF
LFM2.5-Audio-1.5B
Lfm · Mungert
Params1.17B
Context125K
Q4_K_M0.7 GB
Tool use
PA
pipizhao_pandalyst-7b-v1.2
Pandalyst · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
SA
Saiga_Nemo_12B_Sft_M10_D16
Saiga · NightForger
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
GI
Gille StrangeMerges 52 7B dare ties
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 8B LexiFun Uncensored V1
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Roleplay
GR
granite-7b-lab
Granite · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GE
gemma-4-26B-A4B-it
Gemma · sphaela
Params25.2B
Context256K
Q4_K_M15.4 GB
ReasoningTool useVision
QW
Qwen Image Layered
Qwen · vantagewithai
Params20.4B
Context—
Q4_K_M13.1 GB
GGUF
LL
dolphin-2.9.1-llama-3-8b
Llama · crusoeai
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral7B PairRM SPPO Iter3 imatrix
Mistral · duyntnet
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5-1.5B-Instruct
Qwen · Mungert
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
FI
FireRed Image Edit 1.0
Firered · vantagewithai
Params20.4B
Context—
Q4_K_M13.1 GB
GGUF
QW
Ornith 1.0 9B
Qwen · samuelchristlie
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
QW
Vikhr-Qwen-2.5-1.5B-Instruct
Qwen · QuantFactory
Params1.78B
Context32K
Q4_K_M1.1 GB
GGUF
LL
Infinity Instruct 7M Gen Llama3 1 8B
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
IN
InternLM2
InternLM · second-state
Params7.74B
Context32K
Q4_K_M4.7 GB
Coding
XW
Xwin-MLewd-13B-V0.2
Xwin · duyntnet
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
WI
WizardLM-2-7B
Wizardlm · duyntnet
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PF
pfnet nekomata 14b pfn qfin inst merge
Pfnet · mmnga
Params14.2B
Context8K
Q4_K_M9.4 GB
GGUF
GE
Gemma 2 9B It SPPO Iter3
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
IN
InternLM2
InternLM · QuantFactory
Params7.74B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen2 1.5B Ita
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
RA
RakutenAI 7B instruct
Rakutenai · mmnga
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
LL
Llama 3.1 8B Instuct Uz
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing