Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
migtissera Tess v2.5 Phi 3 medium 128k 14B
Phi · RichardErkhov
Params14B
Context128K
Q4_K_M8.6 GB
GGUF
TO
tokyotech llm Swallow 7b instruct hf
Tokyotech · RichardErkhov
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
TH
The Trinity Coder 7B
The · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
LL
Turkish Llama 8b Instruct v0.1
Llama · ytu-ce-cosmos
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
TI
TinySolar-248m-4k-py
Tinysolar · RichardErkhov
Params248M
Context32K
Q4_K_M0.2 GB
GGUF
QW
Qwen3-TTS-12Hz-0.6B-CustomVoice
Qwen · sakasegawa
Params1.02B
Context—
Smallest0.8 GB
GGUF
CO
ConfigurableBeagle-11B
Configurablebeagle · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
YI
Yi 1.5 34B 32K
Yi · RichardErkhov
Params34.4B
Context32K
Q4_K_M20.7 GB
GGUF
MI
speechless-mistral-dolphin-orca-platypus-samantha-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AL
allknowingroger CalmExperiment 7B slerp
Allknowingroger · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
JP
jpacifico Chocolatine 14B Instruct DPO v1.2
Jpacifico · RichardErkhov
Params14B
Context4K
Q4_K_M8.6 GB
GGUF
LL
hon9kon9ize Cantonese Llama 2 7B preview20240625
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
Phi 3 Mini 4k Instruct
Phi · SanctumAI
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
YI
Yi 34B 200K DARE Merge v7
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
LL
PiSSA Llama 2 7b R128
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
OP
Openbuddy Yi1.5 34b v21.3 32k
Openbuddy · RichardErkhov
Params34.4B
Context128K
Q4_K_M20.7 GB
GGUF
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
NeuralHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
FL
PornMaster flux2 klein 9b
Flux · cl0537furry
Params8.19B
Context256K
Q4_K_M5.0 GB
GGUF
CL
ClaudioItaly Evolutionstory 7B v3.0
Claudioitaly · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
YuisekinAI Mistral 0.7B
Mistral · RichardErkhov
Params746M
Context4K
Q4_K_M0.5 GB
GGUF
MI
flammenai flammen15X mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 0.5B Instruct
Qwen · legraphista
Params494M
Context32K
Smallest0.3 GB
GGUF
MI
speechless-mistral-six-in-one-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
KU
Kukedlc NeuralSynthesis 7B v0.3
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AT
Athene 70B
Athene · legraphista
Params70.6B
Context8K
Smallest15.3 GB
GGUF
UM
Umievo Itr012 Gleipnir 7B
Umievo · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
QW
gte-Qwen2-1.5B-instruct
Qwen · RichardErkhov
Params1.78B
Context128K
Q4_K_M1.1 GB
GGUF
ST
StableBeluga 7B
Stablebeluga · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF

Other quants

Keep browsing