Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
maars505 trained qwen2 dpo model5 deepspeed
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
MI
KoboldAI Mistral 7B Erebus v3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.6-35B-A3B-QD
Qwen · QuaduxIT
Params35.5B
Context256K
Smallest10.6 GB
ReasoningTool useVision
SO
SOLAR 10.7B Merge Dpo_V1
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
ST
Opencsg Starcoder2 3b v0.1
StarCoder · RichardErkhov
Params3.18B
Context16K
Q4_K_M2.0 GB
Coding
AR
Arkana08 NIHAPPY L3.1 8B v0.09
Arkana · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
RO
RoMistral-7b-Instruct
Romistral · legraphista
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IM
z image big love v3
Image · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
GGUF
RO
Rohan5076 medquad text generation gpt2
Rohan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
OP
openchat openchat 3.5 1210
Openchat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
ST
Stablelm 2 1_6b
Stablelm · RichardErkhov
Params1.64B
Context4K
Q4_K_M1.0 GB
GGUF
QW
Qwen2.5-0.5B-SFT-1e-5-5ep
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
Tool use
GI
Gille StrangeMerges 7 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Vezora Mistral 14b Merge Base
Mistral · RichardErkhov
Params14.2B
Context32K
Q4_K_M8.6 GB
GGUF
PH
M1LL1X microsoft phi 3 5 mini trained abliterated QA 200 all modules merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama3 ChatQA 1.5 8B
Llama · Monor
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 2 7b Hf Gpt 4 80k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
ME
mergekit community mergekit slerp iyanrea
Mergekit · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Speechless Mistral Hermes Code 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
CodingTool use
ST
finetune_starcoder2_3b_tokens_added
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
CD
CDS CL 7b Instruct Hf 20241002 Pretraining Exp2
Cds · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
MA
Marcoroni Neural Chat 7B v2
Marcoroni · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
MiniMoog Mergerix 7b v0.3
Minimoog · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
peft internal testing tiny dummy qwen2
Qwen · RichardErkhov
Params11M
Context—
Q4_K_M0.0 GB
GGUF
LL
Dumi Llama3.2 3b Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
FA
ssmits Falcon2 5.5B Dutch
Falcon · RichardErkhov
Params5.47B
Context2K
Q4_K_M3.4 GB
GGUF
LL
lianghsun Llama 3.2 Taiwan 3B
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
GGUF
MU
mudler Asinello Minerva 3B v0.1
Mudler · RichardErkhov
Params2.89B
Context16K
Q4_K_M1.8 GB
GGUF
NU
Nutanix checkpoint 1800 merged
Nutanix · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
tinyllama-1.1B-intermediate-step-715k-1.5T-dpo-lora-merged
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF

Other quants

Keep browsing