Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MA
Marco o1
Marco · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB
GGUF
AU
Aurora Nights 70B v1.0
Aurora · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
LL
scb10x llama 3 typhoon v1.5x 70b instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
LL
LLaMA 3 8B SFR Iterative DPO R
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Nous Hermes 2 Mixtral 8x7B SFT
Mistral · NousResearch
Params46.7B
Context32K
Q4_K_M28.4 GB
Tool use
GE
gemma 2 baku 2b it
Gemma · QuantFactory
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GE
gemma-2b
Gemma · mlabonne
Params2.51B
Context8K
Q4_K_M1.5 GB
GGUF
BI
BioMistral 7B
Biomistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LI
OmniLing V1 8b experimental
Lite · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ST
starcoder2-15b-instruct-v0.1
StarCoder · gaianet
Params16B
Context16K
Q4_K_M9.9 GB
Coding
MI
Candle Mistral 7B Instruct v0.1
Mistral · DanielClough
Params7.24B
Context—
Smallest2.4 GB
GGUF
VI
vicgalle ConfigurableHermes 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Llama 3 13B Instruct v0.1 GGUF smashed
Llama · PrunaAI
Params13.3B
Context8K
Q4_K_M8.1 GB
GGUF
AL
AlphaMonarch 7B imatrix
Alphamonarch · duyntnet
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
DeepSeek R1 Distill Llama 8B
DeepSeek · second-state
Params8.03B
Context128K
Q4_K_M4.9 GB
ReasoningTool use
MI
OpenHermes 2.5 Strix Philosophy Mistral 7B LoRA
Mistral · sayhan
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MI
Candle llava v1.6 mistral 7b
Mistral · DanielClough
Params7.57B
Context—
Smallest2.5 GB
GGUF
RE
Refact 1 6 base
Refact · QuantFactory
Params1.59B
Context4K
Q4_K_M1.0 GB
Coding
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
AesCoder-4B
Qwen · Mungert
Params4.41B
Context256K
Q4_K_M3.0 GB
CodingReasoningTool use
PR
PrathameshLLM 2B
Prathameshllm · pmking27
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
SM
SmolLM 1.7B
SmolLM · QuantFactory
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
GE
Gemma 3 12b It
Gemma · second-state
Params11.8B
Context128K
Q4_K_M7.3 GB
Vision
GE
Gemma2 9B TitanFusion
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
TE
TextBase 7B v0.1
Textbase · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
ScaduTorrent1.1 8b model stock
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Roleplay
GT
Gte Multilingual Reranker Base
Gte · gpustack
Params306M
Context8K
Q4_K_M0.2 GB
Embedding
DE
DeepSeek-OCR
DeepSeek · NexaAI
Params2.93B
Context—
Smallest1.7 GB
GGUF
LL
Llama 3 Instruct 8B SimPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ML
MLewd L2 Chat 13B
Mlewd · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF

Other quants

Keep browsing