Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
codellama CodeLlama 13b Python hf
Llama · RichardErkhov
Params13B
Context—
Q4_K_M0.0 GB
Coding
QW
Vikhr-Qwen-2.5-1.5B-Instruct
Qwen · Vikhrmodels
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
GL
Glm Edge 1.5b Chat
GLM · zai-org
Params1.59B
Context2K
Q4_K_M1.0 GB
GGUF
QW
Qwen3-ASR-1.7B
Qwen · sakasegawa
Params2.04B
Context—
Smallest1.2 GB
GGUF
PA
palmyra mini thinking AIO
Palmyra · prithivMLmods
Params1.78B
Context128K
Q4_K_M1.0 GB
CodingReasoning
FL
PornMaster flux2 klein 9b
Flux · placetrtr
Params8.19B
Context256K
Q4_K_M5.0 GB
GGUF
MI
PVD 160k Mistral 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
K2S3 Mistral 7b v1.48
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
OP
OpenELM-450M
Openelm · RichardErkhov
Params457M
Context2K
Q4_K_M0.3 GB
GGUF
QW
Qwen3.5 Uncensored Aggressive
Qwen · tripolskypetr
Params122B
Context256K
Q4_K_M1.3 GB
Vision
SP
speakleash Bielik 7B Instruct v0.1
Speakleash · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
QW
qwen3 embed 8b
Qwen · cstr
Params7.57B
Context40K
Smallest4.4 GB
Embedding
QW
Qwen2.5 3B Instruct
Qwen · legraphista
Params3.09B
Context32K
Smallest0.8 GB
Tool use
PH
Phi 3 medium 4k instruct IMat
Phi · legraphista
Params14B
Context4K
Smallest3.0 GB
GGUF
LL
colossal llama 2 7b base
Llama · shaowenchen
Params7.04B
Context4K
Smallest3.0 GB
GGUF
IS
Isaak Carter J.O.S.I.E.3 Beta9 7B slerp
Isaak · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · legraphista
Params8B
Context128K
Smallest2.0 GB
GGUF
QW
xpo-qwen2
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
MI
Minitron 8B Base
Minitron · legraphista
Params8.27B
Context4K
Smallest3.6 GB
GGUF
LL
tifa benchmark llama2 tifa question generation
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
grimjim Llama 3 Instruct demi merge 8B
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MV
mvpmaster NeuralDareDMistralPro 7b slerp
Mvpmaster · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
aifeifei798 DarkIdol Llama 3.1 8B Instruct 1.0 Uncensored
Llama · RichardErkhov
Params8B
Context—
Q4_K_M0.0 GB
GGUF
QW
gte-Qwen2-7B-instruct
Qwen · RichardErkhov
Params7.61B
Context128K
Q4_K_M4.7 GB
GGUF
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · anonymous12122
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
PH
chrisob94 phi 3 7b dissertation doc chat experiment 10 fold
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
FL
flux 3 image klein base
Flux · leejet
Params5.54B
Context—
Smallest2.3 GB
GGUF
LL
Llama-3-8B-Instruct-64k
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral-7B-Instruct-v0.3
Mistral · SanctumAI
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · legraphista
Params7.62B
Context32K
Smallest1.9 GB
Tool use

Other quants

Keep browsing