Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
EU
EuroLLM 9B Instruct
Eurollm · QuantFactory
Params9.15B
Context4K
Q4_K_M5.6 GB
GGUF
MI
laser dolphin mixtral 2x7b dpo
Mistral · macadeliccc
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
YU
YugoGPT Quantized
Yugogpt · datatab
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.1 SuperNova Lite
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
YO
Youtu LLM 2B
Youtu · AaryanK
Params1.96B
Context128K
Q4_K_M1.2 GB
CodingReasoningTool use
GE
gemma-2b-it
Gemma · second-state
Params2.51B
Context8K
Q4_K_M1.5 GB
GGUF
QW
Qwen2.5 Coder 32B Instruct
Qwen · second-state
Params32.8B
Context32K
Q4_K_M19.9 GB
CodingTool use
LL
saiga llama3 8b
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
Roleplay
LL
Meta Llama 3 8B Instruct abliterated v3
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Tongda1 1.5B BKI
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
LL
ELYZA japanese CodeLlama 7b
Llama · mmnga
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
KA
fpham_karen_theeditor_13b_hf
Karen · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
CO
codegeex4-all-9b
Codegeex · QuantFactory
Params9.4B
Context128K
Q4_K_M6.3 GB
GGUF
GL
Glyph
GLM · Mungert
Params9.4B
Context128K
Q4_K_M6.6 GB
ReasoningTool useVision
GR
Granite 4.0 Micro Base
Granite · ibm-granite
Params3.4B
Context128K
Q4_K_M2.1 GB
GGUF
AI
Airoboros L2 70b 2.2.1
Airoboros · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
MI
MiniCPM V 4.6 Thinking Abliterated MAX
MiniCPM · prithivMLmods
Params752M
Context256K
Q4_K_M0.5 GB
ReasoningTool useVision
TR
Translategemma 4b It
Translategemma · tatsuyaaaaaaa
Params3.88B
Context128K
Q4_K_M2.5 GB
GGUF
SM
SmolLM2 360M Instruct
SmolLM · QuantFactory
Params362M
Context8K
Q4_K_M0.3 GB
GGUF
LI
T lite instruct 0.1
Lite · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · PrunaAI
Params3.61B
Context128K
Q4_K_M2.2 GB
Tool use
NE
neural-chat-7b-v3-3
Neural · duyntnet
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FI
financeocr models
Financeocr · Sevilze
Params891M
Context128K
Q4_K_M0.5 GB
ReasoningTool useVision
VA
VAGOsolutions SauerkrautLM 7b HerO
Vagosolutions · RichardErkhov
Params3.8B
Context—
Q4_K_M2.2 GB
GGUF
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
VI
felixchao_vicuna-33b-coder
Vicuna · TheBloke
Params32.5B
Context2K
Q4_K_M19.6 GB
Coding
LL
microsoft LLaMA 2 7b GTL Delta
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Josiefied Qwen2.5 7B Instruct Abliterated v2
Qwen · Goekdeniz-Guelmez
Params7B
Context32K
Q4_K_M4.7 GB
GGUF
BA
openerotica_basilisk-7b-v0.2
Basilisk · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mixtral 8x7B Instruct v0.1
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF

Other quants

Keep browsing