Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Minerva 3B base RAG
Minerva · QuantFactory
Params2.89B
Context16K
Q4_K_M1.8 GB
GGUF
MI
vagosolutions_sauerkrautlm-mixtral-8x7b-instruct
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
GGUF
XL
xLAM 2 1b Fc R
Xlam · Salesforce
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
GE
Gemma 3 4b It Abliterated
Gemma · matrixportalx
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
LL
Llama-3-8b-64k-PoSE
Llama · LiteLLMs
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Hermes 2 Pro Mistral 7B
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
PE
PersianMind v1.0
Persianmind · QuantFactory
Params6.8B
Context2K
Q4_K_M4.1 GB
GGUF
MI
mathstral 7B v0.1
Mistral · QuantFactory
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
LL
tifa benchmark llama2 tifa question generation
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
grimjim Llama 3 Instruct demi merge 8B
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
JU
fblgit_juanako-7b-v1
Juanako · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MV
mvpmaster NeuralDareDMistralPro 7b slerp
Mvpmaster · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
AMD Llama 135m
Llama · QuantFactory
Params134M
Context2K
Q4_K_M0.1 GB
GGUF
LL
aifeifei798 DarkIdol Llama 3.1 8B Instruct 1.0 Uncensored
Llama · RichardErkhov
Params8B
Context—
Q4_K_M0.0 GB
GGUF
YI
Yi 6B
Yi · TheBloke
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
GE
C2S-Scale-Gemma-2-27B
Gemma · Mungert
Params27.2B
Context8K
Q4_K_M16.7 GB
GGUF
OP
openchat 3.6 8b 20240522
Openchat · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
ST
stable code 3b
Stable · QuantFactory
Params2.8B
Context16K
Q4_K_M1.7 GB
Coding
AL
totally-not-an-llm_alpacacielo2-7b-8k
Alpacacielo · TheBloke
Params6.74B
Context8K
Q4_K_M4.1 GB
GGUF
LL
Meta-Llama-3-8B-Instruct
Llama · duyntnet
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
CO
cohere transcribe ja v0.1
Cohere · TransWithAI
Params2.07B
Context—
Smallest1.5 GB
GGUF
QW
gte-Qwen2-7B-instruct
Qwen · RichardErkhov
Params7.61B
Context128K
Q4_K_M4.7 GB
GGUF
HA
Harmonic 9B
Harmonic · DJLougen
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
MI
SlimHercules-4.0-Mistral-7B-v0.2
Mistral · duyntnet
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IQ
IQuest Coder V1 40B Instruct
Iquest · AaryanK
Params39.8B
Context128K
Q4_K_M24.0 GB
CodingTool use
PH
chrisob94 phi 3 7b dissertation doc chat experiment 10 fold
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
LL
meta-llama-13b
Llama · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
LL
Meta Llama 3 70B Instruct
Llama · QuantFactory
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
WE
petrogpt_westseverus-7b-dpo
Westseverus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BL
Blue Orchid 2x7b
Blue · QuantFactory
Params12.9B
Context8K
Q4_K_M7.8 GB
GGUF

Other quants

Keep browsing