Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
duy nq phi 3 mini 4k merge v4
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
MI
yjwon mp mistral7b sft1 b512 lr5e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
BFS Search mistral multi rel I2B2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama-2-7b-instruct
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
GP
Gpt2 Large
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
LL
Llama-2-13b-chat-longlora-32k-sft
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
mlfoundations dev mistral alpaca sft sample 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CH
ChatAYT Lora Assamble Marcoroni
Chatayt · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MA
Magpie Align MagpieLM 8B SFT v0.1
Magpie · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
KO
KoSOLAR-10.7B-v0.2
Kosolar · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
LL
LLama 3.1 8B HyperNova
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
EL
Eleuther Pythia70M Hh Dpo
Eleuther · RichardErkhov
Params70M
Context2K
Q4_K_M0.0 GB
GGUF
LL
Llama 3.2 3b It Ecommerce ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
GR
Iq Code Evmind v1 Granite 8b Instruct
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
Coding
LL
mlfoundations dev llama2 oh teknium scaling down random 0.5
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
ED
edbeeching EleutherAI pythia 2.8b
Edbeeching · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
GP
Gpt2 Cia World Factbook
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
ZE
zephyr-beta-Nebula-v2-7B
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
DeepFeatTextEmbeddingLarge
Deepfeattextembeddinglarge · RichardErkhov
Params1.78B
Context128K
Q4_K_M1.1 GB
Embedding
ZE
zephyr-7b-dpo-full-judge-before
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 1B Instruct Zh De Block
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
QW
psneto qwen2 arithmetic 2T FT
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
IN
interview eval zephyr 7b stem case 7
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
LL
verbalyze Llama3.1 finetuned
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF

Other quants

Keep browsing