Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
OL
OLMo 2 0425 1B Instruct
OLMo · allenai
Params1.48B
Context4K
Q4_K_M0.9 GB
GGUF
MI
NeuralHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Meta Llama 3.1 8B Instruct
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
WA
Wan2.2 TI2V 5B
Wan · squirrelae
Params5B
Context—
Q4_K_M3.4 GB
GGUF
PI
ddh0_pirouette-7b
Pirouette · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AE
Aetheria L2 70B
Aetheria · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
RA
RakutenAI 2.0 Mini Instruct
Rakutenai · mmnga
Params1.53B
Context128K
Q4_K_M0.9 GB
GGUF
YI
dolphin-2.9.3-Yi-1.5-34B-32k
Yi · crusoeai
Params34.4B
Context32K
Q4_K_M20.7 GB
GGUF
ST
starcoder2-3b
StarCoder · second-state
Params3.03B
Context16K
Q4_K_M1.8 GB
Coding
GR
granite-4.0-micro-base
Granite · Mungert
Params3.4B
Context128K
Q4_K_M2.1 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · duyntnet
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
GL
GLM 4.7 Flash
GLM · evilfreelancer
Params29.9B
Context198K
Q4_K_M18.1 GB
ReasoningTool use
SO
SOLAR 10.7B Instruct v1.0
Solar · QuantFactory
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
UN
UndiMix v2 13B
Undimix · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
QW
Qwen Image Edit 2511
Qwen · w3ss
Params20.4B
Context—
Q4_K_M13.2 GB
GGUF
LE
leolm_leo-hessianai-13b-chat
Leo · TheBloke
Params13B
Context8K
Q4_K_M7.9 GB
GGUF
FL
Flux Fusion DS merge gguf nf4 fp4 fp8 fp16
Flux · Anibaaal
Params11.9B
Context—
Q4_K_M6.9 GB
GGUF
LL
Llava v1.6 Vicuna 7B
LLaVA · second-state
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
QW
Qwen2.5 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
GGUF
CL
ClaudioItaly Evolutionstory 7B v3.0
Claudioitaly · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
YuisekinAI Mistral 0.7B
Mistral · RichardErkhov
Params746M
Context4K
Q4_K_M0.5 GB
GGUF
AI
airoboros l2 13B 2.2.1
Airoboros · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
flammenai flammen15X mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CO
Yi Coder 1.5B
Coder · QuantFactory
Params1.48B
Context128K
Q4_K_M1.0 GB
Coding
GE
Huihui Gemma 4 26B A4B It Abliterated
Gemma · bullerwins
Params25.2B
Context256K
Q4_K_M16.8 GB
GGUF
QW
Qwen2.5 VL 7B Instruct
Qwen · second-state
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
DE
DeepSeek-Coder-V2-Lite-Instruct
DeepSeek · second-state
Params15.7B
Context160K
Q4_K_M10.4 GB
Coding
LI
Ling-3.0-tiny-base-30T
Ling · FriskyFennec
Params8.2B
Context8K
Q4_K_M5.0 GB
GGUF
SY
synthia-70b-v1.1
Synthia · TheBloke
Params69B
Context2K
Q4_K_M41.4 GB
GGUF
LL
LLaMA2 13B Tiefighter
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF

Other quants

Keep browsing