Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Unichat llama3 Chinese 8B 28K
Llama · Monor
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
NO
nota ai st vicuna v1.3 10.5b ppl
Nota · RichardErkhov
Params10.5B
Context2K
Q4_K_M6.3 GB
GGUF
TR
Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B
Truthful · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
IB
ibivibiv aegolius acadicus v1 30b
Ibivibiv · RichardErkhov
Params29.8B
Context32K
Q4_K_M18.0 GB
GGUF
MI
Minerva
Minerva · RichardErkhov
Params2.89B
Context16K
Q4_K_M1.8 GB
GGUF
GE
Gemma 2 9b Chatml
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
saqada Llama 3.1 8B with 16bit merged lora adapters MOV v7
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
QuantumIntelligence QI mistral 7B slerp
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
deepseek-coder-33b-instruct
DeepSeek · RichardErkhov
Params33.3B
Context16K
Q4_K_M19.9 GB
Coding
JE
Jellyfish 8B
Jellyfish · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ML
mlfoundations dev stackoverflow 10000tasks 1p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
RLHFlow LLaMA3 iterative DPO final
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
YC
ycros BagelMIsteryTour 8x7B
Ycros · RichardErkhov
Params57.6B
Context32K
Q4_K_M28.4 GB
GGUF
LL
LLaMAntino 2 Chat 7b Hf UltraChat ITA
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
AM
amazingvince Not WizardLM 2 7B
Amazingvince · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced
Gemma · kai2336
Params25.2B
Context256K
Q4_K_M16.8 GB
ReasoningTool useVision
MI
E5 R Mistral 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BR
BroAlanTaps GPT2 large 256 16500steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
VO
Voxtral Mini 3B 2507
Voxtral · Bercraft
Params4.68B
Context—
Smallest1.6 GB
GGUF
RE
reflect_llm8B_Om2SftT1-Om2G8kOm2AgIpsdpIter1T1_b1.0
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
ML
mlfoundations dev stackoverflow 5000tasks .75p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
TI
Tiny-Pirate-1.1b-v0.1
Tiny · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
FO
Foundation Sec 8B Instruct GGUF
Foundation · 2p8xx
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
YI
dolphin-2.9.1-yi-1.5-34b
Yi · RichardErkhov
Params34.4B
Context8K
Q4_K_M20.7 GB
GGUF
LL
BramVanroy llama2 13b ft mc4 nl cleaned tiny
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
LDCC Instruct Llama 2 Ko 13B v4.1.2
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
FU
FusionNet
Fusionnet · RichardErkhov
Params21.2B
Context4K
Q4_K_M12.7 GB
GGUF
DI
Diksha2001 VLLM new
Diksha · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing