Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
TI
stories converted
Tinyllamas · shibatch
Params24M
Context128
Smallest0.0 GB
GGUF
SE
semcoder semcoder 1030
Semcoder · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
VI
vicuna 33b v1.3
Vicuna · shaowenchen
Params32.5B
Context2K
Smallest13.5 GB
GGUF
QW
Qwen3 4B Thinking 2507 Unsloth
Qwen · magiccodingman
Params4.02B
Context256K
Q4_K_M2.5 GB
ReasoningTool use
PH
IntelLabs sqft phi 3 mini 4k 30 base
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
Unichat llama3 Chinese 8B
Llama · Monor
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
leap llm Meta Llama 3 8B Instruct sft alfworld iter2
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Papy_2_Llama 3.1 8B Instruct_text
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MA
Matter 0.1 7B Boost
Matter · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 2B
Qwen · ZuzeTt
Params1.88B
Context256K
Q4_K_M1.3 GB
ReasoningTool useVision
TI
TinyLlama 1.1B intermediate step 1431k 3T
Tinyllama · andrijdavid
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
MA
magnum v1 32b
Magnum · anthracite-org
Params32.5B
Context32K
Smallest10.3 GB
GGUF
PH
tuong nguyen prd Phi 3.5 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
SA
SambaLingo Arabic Base
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
GGUF
BI
BioMistral BioMistral 7B TIES
Biomistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RO
roadz dv finetuned 211124
Roadz · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
liyimengwork llama 3.1 8B wenyanwen2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
GE
Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
LI
lightblue reranker 0.5 bincont filt
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
EmbeddingTool use
QW
Qwen3.5 0.8B
Qwen · titan0115
Params752M
Context256K
Q4_K_M0.5 GB
ReasoningTool use
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · IMUGLYHUH
Params1.86B
Context256K
Smallest0.9 GB
GGUF
GL
GLM 4.6 Derestricted
GLM · ReadyArt
Params357B
Context198K
Smallest0.7 GB
GGUF
PY
Pygmalion 3 12b
Pygmalion · PygmalionAI
Params12.2B
Context1000K
Smallest4.9 GB
GGUF
LL
DarkIdol Llama 3.1 8B Instruct 1.2 Uncensored
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Lunavox Qwen3 TTS
Qwen · wkwong
Params142M
Context64K
Smallest0.2 GB
GGUF
SA
saishf SOVLish Maid L3 8B
Saishf · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PR
PranavHarshan NarraLarge
Pranavharshan · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
CO
Yi Coder 9B Chat Instruct TIES
Coder · RichardErkhov
Params28.3B
Context128K
Q4_K_M17.1 GB
Coding
LL
namespace Pt Llama 3 8B Instruct 80K QLoRA Merged
Llama · RichardErkhov
Params8.03B
Context80K
Q4_K_M4.9 GB
GGUF
YI
Yi-1.5-9B-Chat
Yi · RichardErkhov
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF

Other quants

Keep browsing