Q5_0 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
IN
InternLM2StepProver
InternLM · QuantFactory
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
TY
Typhoon 7b Instruct 02 19 2024
Typhoon · RichardErkhov
Params7.27B
Context32K
Q4_K_M4.4 GB
GGUF
BE
bella bartender heretic 1b
Bella · juiceb0xc0de
Params1.24B
Context128K
Smallest0.6 GB
Tool use
LL
LLaMA 2 7B 32K
Llama · rozek
Params6.74B
Context32K
Q4_K_M4.1 GB
GGUF
CH
MN Chthonia 12B i1
Chthonia · BasedAGI
Params12.2B
Context128K
Q4_K_M7.5 GB
Tool use
QW
Reasoning 0.5b
Qwen · QuantFactory
Params494M
Context32K
Q4_K_M0.4 GB
Reasoning
PH
Phi Mini MoE
Phi · gabriellarson
Params7.65B
Context4K
Q4_K_M5.0 GB
GGUF
GE
Gemma 4 E4B It Qat Heretic_Decensored
Gemma · prithivMLmods
Params7.46B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
MI
OpenHermes 2.5 Mistral 7B
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MI
MistarlingMaid 2x7B Base
Mistarlingmaid · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
WI
WizardCoder Python 7B v1.0
Wizardcoder · second-state
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
Nous Hermes 2 Mistral 7B DPO
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
QW
Qwen2.5 Coder 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
LL
Llama-3-Open-Ko-8B-Instruct-preview
Llama · teddylee777
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
BR
Breeze 7B 32k Base V1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
GGUF
DE
NuminaMath 7B CoT
DeepSeek · QuantFactory
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
HE
HelpingAI Lite
Helpingai · QuantFactory
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
NE
Nethena MLewd Xwin 23B
Nethena · TheBloke
Params22.8B
Context4K
Q4_K_M13.8 GB
GGUF
WI
WizardLM-13B-V1.2
Wizardlm · duyntnet
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
GL
glm-4-9b-chat
GLM · duyntnet
Params9.4B
Context128K
Q4_K_M6.3 GB
Tool use
LL
Llama 3 Yggdrasil 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
RW
Rwkv7 G1 1.5b Instruct Preview
Rwkv · mzwing
Params1.53B
Context1024K
Q4_K_M1.0 GB
GGUF
LL
Llama 3 Groq 8B Tool Use
Llama · QuantFactory
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
GE
genesis-4B-DEMO
Genesis · epsill0n
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
IN
InstructLM 500M
Instructlm · QuantFactory
Params568M
Context2K
Q4_K_M0.4 GB
GGUF
LL
FireStorm Llama 3.1 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Artificium llama3.1 8B 001
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
GL
Glm Edge V 5b
GLM · zai-org
Params4.33B
Context3K
Q4_K_M2.6 GB
Vision
LL
Hermes 3 Llama 3.1 70B
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use

Other quants

Keep browsing