Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
IN
Inv Konstanta V4 Alpha 7B
Inv · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
LL
llama 2 7b chat catala
Llama · xaviviro
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Nous Hermes 2 Mixtral 8x7B DPO
Mistral · NousResearch
Params46.7B
Context32K
Q4_K_M28.4 GB
Tool use
VI
lmsys_vicuna-13b-v1.5-16k
Vicuna · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
GGUF
ST
Starcoder2 7b
StarCoder · QuantFactory
Params7.17B
Context16K
Q4_K_M4.5 GB
Coding
ME
epfl-llm_meditron-7b
Meditron · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
VI
vibevoice
Vibevoice · gguf-org
Params2.7B
Context—
Q4_K_M3.5 GB
GGUF
MI
mistralai_mistral-7b-instruct-v0.1
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GL
Huihui GLM 4.6V Flash Abliterated
GLM · seanbailey518
Params9.4B
Context128K
Q4_K_M6.2 GB
Vision
FL
FLUX.2 klein 9B
Flux · drends
Params9.08B
Context—
Q4_K_M5.9 GB
GGUF
OP
OpenELM-270M-Instruct
Openelm · RichardErkhov
Params272M
Context2K
Q4_K_M0.2 GB
GGUF
LL
nkpz_llama2-22b-daydreamer-v3
Llama · TheBloke
Params21.8B
Context4K
Q4_K_M13.2 GB
GGUF
MI
macadeliccc_laser-dolphin-mixtral-2x7b-dpo
Mistral · TheBloke
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
CH
Chinese Alpaca 2 13B
Chinese · TheBloke
Params13.3B
Context4K
Q4_K_M8.0 GB
GGUF
CO
Code 290k 13B
Code · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
Coding
TO
llm-agents_tora-code-13b-v1.0
Tora · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
Coding
DE
DeepSeek R1 Distill Qwen 14B
DeepSeek · second-state
Params14.8B
Context128K
Q4_K_M9.0 GB
Reasoning
LL
Meta Llama 3 70B Instruct
Llama · mmnga
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
MI
jphme_em_german_mistral_v01
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LO
LongAlpaca 70B
Longalpaca · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
LL
Taigi Llama 2 Translator 7B
Llama · RichardErkhov
Params6.94B
Context4K
Q4_K_M4.2 GB
GGUF
PR
prometheus 7B v1.0
Prometheus · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
PH
Phi 3 mini 4k geminified
Phi · QuantFactory
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
QW
Qwen3.5 9B
Qwen · prithivMLmods
Params9.2B
Context256K
Q4_K_M5.8 GB
Vision
ME
meditron 70B
Meditron · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
LL
Llama3 German 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama OuteTTS 1.0 1B
Llama · OuteAI
Params1.25B
Context128K
Q4_K_M0.8 GB
GGUF
RE
ReflectionCoder DS 33B
Reflectioncoder · RichardErkhov
Params33.3B
Context16K
Q4_K_M19.9 GB
Coding
LL
Llava v1.5 7B
LLaVA · second-state
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
LL
Llama 3 Alpha Ko 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing