Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
ST
starcoderbase 7b
StarCoder · QuantFactory
Params7.5B
Context8K
Q4_K_M4.9 GB
Coding
OC
octo net
Octo · NexaAI
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
BO
Bookworm 10.7B v0.4 DPO
Bookworm · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
MA
MAmmoTH 7B
Mammoth · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GI
GigaChat 20B A3B instruct v1.5
Gigachat · ai-sage
Params20.6B
Context128K
Q4_K_M12.5 GB
GGUF
NE
NeuTTS Air
Neutts · BricksDisplay
Params262M
Context—
Q4_K_M0.3 GB
GGUF
LL
DarkIdol Llama 3.1 8B Instruct 1.0 Uncensored
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
VI
VibeThinker 3B
Vibethinker · prithivMLmods
Params3.09B
Context128K
Q4_K_M1.9 GB
CodingReasoning
FL
flux dev
Flux · gguf-org
Params123M
Context—
Q4_K_M6.7 GB
GGUF
QW
VulnLLM-R-7B
Qwen · Mungert
Params7.6B
Context32K
Q4_K_M4.9 GB
CodingReasoning
DO
Dolphin 2.1 70B
Dolphin · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
DE
DeepSeek Coder V2 Lite Base
DeepSeek · QuantFactory
Params15.7B
Context160K
Q4_K_M10.4 GB
Coding
IN
Inkbot 13B 4k
Inkbot · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
DO
dopeystableplats 3b v1
Dopeystableplats · TheBloke
Params2.8B
Context4K
Q4_K_M1.7 GB
GGUF
SA
Sarashina2.2 1b Instruct v0.1
Sarashina · mmnga
Params1.41B
Context8K
Q4_K_M0.9 GB
GGUF
QW
SuperCorrect 7B
Qwen · QuantFactory
Params7.6B
Context4K
Q4_K_M4.7 GB
GGUF
MI
yeungnlp_firefly-mixtral-8x7b
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
GGUF
MI
dolphin 2.8 mistral 7b v02
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
openbuddy_openbuddy-mixtral-8x7b-v15.1
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.5 GB
GGUF
LL
Llama 2 7B vietnamese 20k
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
flammenai flammen20 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RI
RigoChat 7b v2
Rigochat · IIC
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
OP
OpenELM-450M-Instruct
Openelm · RichardErkhov
Params457M
Context2K
Q4_K_M0.3 GB
GGUF
LL
Dolphin 3.0 Llama 3.2 1b
Llama · QuantFactory
Params1.5B
Context128K
Q4_K_M1.0 GB
CodingTool use
LL
OrionStar Yi 34B Chat Llama
Llama · TheBloke
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
MI
Mistral Trismegistus 7B
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MO
Motif Video 2B
Motif · Motif-Technologies
Params2.04B
Context—
Q4_K_M1.2 GB
GGUF
GR
Granite Guardian 4.1 8b
Granite · mrutkows
Params8.38B
Context128K
Q4_K_M5.1 GB
Tool use
MO
moss transcribe.cpp
Moss · mudler
Params909M
Context—
Smallest0.5 GB
GGUF
QW
Qwen2.5 14B Instruct
Qwen · QuantFactory
Params14.8B
Context32K
Q4_K_M9.0 GB
GGUF

Other quants

Keep browsing