Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
LL
Llama3.2 3B Enigma
Llama · QuantFactory
Params3.21B
Context128K
Q4_K_M2.0 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · ThomasBaruzier
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
grimjim Mistral Starling merge trial3 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
OL
Olmo-3-7B-Think
OLMo · Mungert
Params7.3B
Context64K
Q4_K_M4.7 GB
MI
philosophy mistral
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
AP
crumb_apricot-wildflower-20
Apricot · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
FA
Falcon H1 1.5B Deep Instruct
Falcon · tiiuae
Params1.55B
Context128K
Q4_K_M0.9 GB
MA
MAmmoTH2 7B
Mammoth · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
MO
MOSS Image
Moss · PALLER-NOVUS
Params5.08B
Context—
Smallest1.2 GB
ME
MediX R1 8B
Medix · MBZUAI
Params8.19B
Context256K
Q4_K_M5.0 GB
LL
Llama 3.2 3B Instruct
Llama · QuantFactory
Params3.21B
Context128K
Q4_K_M2.0 GB
MI
MiniCPM5 2B
MiniCPM · NANI-Nithin
Params2.52B
Context128K
Q4_K_M1.6 GB
GH
Ghost 7b Alpha
Ghost · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
hf 100 Llama 3.1 Spellbound StoryWriter 0.1 vllm 16bit
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
MI
Mistral Small 24B Instruct 2501
Mistral · BasedAGI
Params23.6B
Context32K
Q4_K_M14.3 GB
PE
Peach 9B 8k Roleplay
Peach · QuantFactory
Params8.8B
Context16K
Q4_K_M5.3 GB
AL
ALMA 13B
Alma · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
XW
Xwin LM 13B v0.2
Xwin · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LL
open-orca_llongorca-13b-16k
Llongorca · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
QW
Qwen1.5-4B-Chat-AWQ-fp16
Qwen · Qwen
Params3.95B
Context32K
Q4_K_M2.5 GB
MY
L2 MythoMax22b Instruct Falseblock
Mythomax · TheBloke
Params21.8B
Context4K
Q4_K_M13.2 GB
WH
whisper medium
Whisper · oxide-lab
Params764M
Context—
Smallest0.3 GB
YI
ChatAllInOne Yi 34B 200K V1
Yi · second-state
Params34.4B
Context195K
Q4_K_M20.7 GB
LL
LongWriter Llama3.1 8b
Llama · duyntnet
Params8B
Context128K
Q4_K_M4.9 GB
LL
Llama 3.1 8B Instruct
Llama · gaianet
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
llama 3 sqlcoder 8b
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
DE
DeepScaleR 1.5B Preview
DeepSeek · QuantFactory
Params1.78B
Context128K
Q4_K_M1.1 GB
MI
brillibits_instruct_mixtral-8x7b-v0.1_dolly15k
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
LL
emma 500 llama2 7b
Llama · QuantFactory
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
azure99_blossom-v3-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB