Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
SP
Spark-X2.6-4B
Spark · Mungert
Params4.11B
Context1024K
Q4_K_M2.6 GB
ReasoningTool use
TI
Tinyllama 15M
Tinyllama · QuantFactory
Params15M
Context256
Q4_K_M0.0 GB
GGUF
ST
stt en fastconformer ctc large
Stt · cstr
Params116M
Context—
Smallest0.1 GB
GGUF
XW
Xwin LM 70B V0.1
Xwin · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
GR
granite 3b code base 128k
Granite · QuantFactory
Params3.48B
Context125K
Q4_K_M2.1 GB
Coding
BA
internistai base 7b v0.2
Base · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
intel_neural-chat-7b-v3-2
Neural · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NO
Noromaid 13B v0.3
Noromaid · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · second-state
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
LL
bavest_fin-llama-33b-merged
Llama · TheBloke
Params32.5B
Context2K
Q4_K_M19.6 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
Starling LM 7B beta
Starling · QuantFactory
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
SP
Spark-X2.5-1.7B
Spark · Mungert
Params1.71B
Context1024K
Q4_K_M1.1 GB
GGUF
DA
abideen_darevox-7b
Darevox · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
YI
Yi-6B-200K
Yi · duyntnet
Params6.06B
Context195K
Q4_K_M3.7 GB
GGUF
AN
Anima 2.9B
Anima · vanes430
Params5.84B
Context—
Smallest3.3 GB
GGUF
TU
allenai_tulu-2-dpo-7b
Tulu · TheBloke
Params6.74B
Context8K
Q4_K_M4.1 GB
GGUF
UC
UCLA AGI zephyr 7b sft full SPIN iter0
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
flozi00_llama-2-13b-german-assistant-v4
Llama · TheBloke
Params13.1B
Context2K
Q4_K_M7.9 GB
GGUF
MI
openpipe_mistral-ft-optimized-1227
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
Starcoder2 3b
StarCoder · QuantFactory
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
MI
mrm8488 mistral 7b ft AgentInstruct
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2 72B Instruct
Qwen · Qwen
Params72.7B
Context32K
Q4_K_M47.4 GB
GGUF
LL
ShiningValiant 1.2
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
BA
bagel
Bagel · calcuis
Params14.6B
Context—
Q4_K_M8.9 GB
GGUF
MI
uukuguy_speechless-mistral-moloras-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SP
uukuguy_speechless-tora-code-7b-v1.0
Speechless · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
PH
Phi 3 mini 4k instruct
Phi · QuantFactory
Params3.82B
Context4K
Q4_K_M2.3 GB
Coding
MI
Dolphin 2.7 mixtral 8x7b
Mistral · second-state
Params46.7B
Context32K
Q4_K_M26.4 GB
CodingTool use
OP
s4sch_open-hermes-2.5-neural-chat-3.1-frankenmerge-11b
Open · TheBloke
Params11.4B
Context32K
Q4_K_M6.8 GB
Tool use

Other quants

Keep browsing