Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama 3.2 Adr Full Model
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
mlfoundations dev llama2 oh teknium scaling down random 0.8
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
ME
mergekit community mergekit slerp janrfan
Mergekit · RichardErkhov
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 3 ELYZA JP 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh v3.1 wo unnatural instructions
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
AY
aya 101
Aya · kcoopermiller
Params12.9B
Context—
Smallest4.2 GB
GGUF
QW
JeloH qwen textgen model9
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
LL
Llama 2 13b Hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
llama2-7b-chat-hf-guanaco
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama3.1 8B ShiningValiant2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
EE
EEVE Korean 10.8B RAFT
Eeve · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
PH
Phi 3.5 Mini Instruct SFT D_chosen Orca Skywork Capybara Infinity Mid
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
gajesh llama 3 2 1b instruct eigentuned
Llama · RichardErkhov
Params14M
Context—
Q4_K_M0.0 GB
GGUF
JI
jina v5 small
Jina · cstr
Params596M
Context32K
Smallest0.4 GB
Embedding
LL
ToolLlama Llama 3 8B
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
ML
mlfoundations dev oh v3 1 only evol instruct 70k
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GI
GitBag reasoning rebel iter 2 1731041913 eta 1e3 lr 3e 7 1731243878
Gitbag · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Reasoning
MI
Nexus IKM Mistral Instruct v0.2 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Voxtral Mini 4B Realtime 2602
Mistral · Bercraft
Params4.43B
Context—
Smallest1.5 GB
GGUF
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
WE
Web Chunk Refining Lm
Web · RichardErkhov
Params354M
Context2K
Q4_K_M0.2 GB
GGUF
OM
omnineura omni OpenELM finetune llm1
Omnineura · RichardErkhov
Params3.04B
Context2K
Q4_K_M1.9 GB
GGUF
AL
aloobun Cypher 7B
Aloobun · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FA
tiiuae falcon mamba 7b instruct
Falcon · RichardErkhov
Params7.27B
Context1024K
Q4_K_M4.2 GB
GGUF
MI
shleeeee mistral ko OpenOrca Platypus v2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B Instruct SLiC HF
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10 minlr1e 6 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
Magpie Align Llama 3 8B WizardLM 196K
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF

Other quants

Keep browsing