Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
FA
Falcon H1 1.5B Deep Instruct
Falcon · tiiuae
Params1.55B
Context128K
Q4_K_M0.9 GB
GGUF
MO
MOSS Image
Moss · PALLER-NOVUS
Params5.08B
Context—
Smallest1.2 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · longsesh
Params34.7B
Context256K
Q4_K_M21.2 GB
Vision
LL
hf 100 Llama 3.1 Spellbound StoryWriter 0.1 vllm 16bit
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
WH
whisper medium
Whisper · oxide-lab
Params764M
Context—
Smallest0.3 GB
GGUF
YI
Yi-9B-Coder
Yi · legraphista
Params8.83B
Context4K
Smallest2.0 GB
Coding
PH
Phi 3 mini 4k instruct IMat
Phi · legraphista
Params3.82B
Context4K
Smallest0.8 GB
GGUF
SM
SmallThinker 4BA0.6B Instruct
Smallthinker · Tiiny
Params4.27B
Context32K
Smallest1.9 GB
GGUF
CH
Chunky-Lemon-Cookie-11B
Chunky · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct Abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
QW
Qwen Image 2.1 Turbo Uncensored
Qwen · suryatmodulus
Params8.19B
Context256K
Smallest3.3 GB
CodingTool useVision
SZ
szymonrucinski Curie 7B v1
Szymonrucinski · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
EV
Evolutionstory
Evolutionstory · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
Tinystories Lay8 Hs512 Hd8 33M
Tinystories · RichardErkhov
Params165M
Context512
Q4_K_M0.1 GB
GGUF
IN
Internlm2_5 1_8b Chat
InternLM · legraphista
Params1.89B
Context32K
Smallest0.5 GB
GGUF
LL
Llama-3-8B-grammar-correction
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
ORLM LLaMA 3 8B
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 7B Instruct Abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
MI
K2S3 Mistral 7b v1.4
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
IC
ichigoberry MonarchPipe 7B slerp
Ichigoberry · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
MI
flammenai flammen18 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ML
mlfoundations dev stackoverflow 5000tasks 0p
Mlfoundations · RichardErkhov
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GE
Gemma 2 2b It
Gemma · legraphista
Params2.61B
Context8K
Smallest0.8 GB
GGUF
MA
MaidStarling 2x7B Base
Maidstarling · RichardErkhov
Params12.9B
Context8K
Q4_K_M7.8 GB
GGUF
OP
OpenELM 3B Instruct
Openelm · RichardErkhov
Params3.04B
Context2K
Q4_K_M1.9 GB
GGUF
CO
Yi Coder 1.5B Chat
Coder · RichardErkhov
Params1.48B
Context128K
Q4_K_M1.0 GB
Coding
LL
ZhangShenao SELM Llama 3 8B Instruct iter 2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
lemon mint gemma ko 7b it v0.40
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF

Other quants

Keep browsing