Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
FL
PornMaster Klein 9b
Flux · rectangleworm
Params8.19B
Context256K
Q4_K_M5.0 GB
GGUF
ZO
ZONOS2
Zonos · Zyphra
Params64M
Context—
Smallest0.0 GB
GGUF
GE
Gemmasutra Mini 2B v1
Gemma · TheDrummer
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
PH
phi 4
Phi · microsoft
Params14.7B
Context16K
Smallest3.6 GB
Coding
OP
openchat-3.6-8b-20240522
Openchat · legraphista
Params8B
Context8K
Smallest2.0 GB
GGUF
GE
avemio digital gemma 2b Sft pretrain 6000 steps merge
Gemma · RichardErkhov
Params2.5B
Context8K
Q4_K_M1.6 GB
GGUF
GE
Gemma 2 27b It
Gemma · legraphista
Params27.2B
Context8K
Smallest6.1 GB
GGUF
NE
Huihui NeoHorse 1 4B Abliterated MTP
Neohorse · huihui-ai
Params4.33B
Context256K
Smallest2.3 GB
ReasoningTool useVision
PH
gretelai Phi 3 mini 128k instruct
Phi · RichardErkhov
Params1.9B
Context—
Q4_K_M0.0 GB
GGUF
QW
qwen3 asr 1.7b
Qwen · cstr
Params2.35B
Context—
Smallest1.5 GB
GGUF
LL
tachyphylaxis Meta Llama 3.1 Chat Uncensored
Llama · RichardErkhov
Params15M
Context—
Q4_K_M0.0 GB
GGUF
LL
KISTI KONI KONI Llama3 8B Instruct 20240729
Llama · RichardErkhov
Params8B
Context—
Q4_K_M0.0 GB
GGUF
MO
Moody Krea 2 Mix V4
Moody · Henrysk
Params12.8B
Context—
Smallest2.5 GB
GGUF
PH
Phi 3 mini 4k instruct update2024 07 03 IMat
Phi · legraphista
Params3.82B
Context4K
Smallest0.8 GB
Coding
TE
test gguf moe sample
Test · SzymonOzog
Params403M
Context—
Smallest0.1 GB
GGUF
LL
ggml llava v1.5 7b
LLaVA · mys
Params6.74B
Context4K
Smallest4.1 GB
Vision
NE
Nemotron3-Nano-4B-Uncensored-HauhauCS-Aggressive
Nemotron · HauhauCS
Params3.97B
Context1024K
Q4_K_M2.8 GB
GGUF
ID
ideogram 4
Ideogram · rectangleworm
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
QW
Qwen3.8 Flash Next MTP
Qwen · drluoto
Params3.9B
Context256K
Q4_K_M2.8 GB
GGUF
CO
Yi Coder 9B Chat
Coder · RichardErkhov
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
QW
Qwen3-TTS-12Hz-1.7B-CustomVoice
Qwen · sakasegawa
Params2.03B
Context—
Smallest1.3 GB
GGUF
MI
maywell Mistral ko 7B v0.1
Mistral · RichardErkhov
Params1.5B
Context—
Q4_K_M0.0 GB
GGUF
DE
DeepSeek-V2-Lite-Chat
DeepSeek · legraphista
Params15.7B
Context160K
Smallest5.0 GB
GGUF
CO
Yi Coder 9B
Coder · RichardErkhov
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
RO
RoMistral-7b-Instruct
Romistral · legraphista
Params7.24B
Context32K
Smallest1.6 GB
GGUF
YI
Yi 6B Chat
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
YU
quantized-yue2-3b
Yue · ngquocvinh
Params3.63B
Context—
Q4_K_M2.9 GB
GGUF
LL
turboderp Cat Llama 3 70B instruct
Llama · RichardErkhov
Params3.3B
Context—
Q4_K_M0.0 GB
GGUF
GL
glm-4-9b-chat
GLM · legraphista
Params9.4B
Context128K
Smallest4.0 GB
GGUF
FI
Finch 3b
Finch · latestissue
Params3.1B
Context1024K
Smallest1.2 GB
GGUF

Other quants

Keep browsing