Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama 3 Instruct 8B CPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ME
MexIvanov MistRAG 7B ruen v1 merged
Mexivanov · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
MI
avemio digital mistral 7b Sft orpo 3000 steps merge
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ZE
Zenith 7B Dpo
Zenith · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
codellama CodeLlama 13b Instruct hf
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
QW
ReAligned Qwen3.5 4B
Qwen · Lazarus-Ai
Params4.21B
Context256K
Smallest1.8 GB
ReasoningTool use
DA
darkc0de BuddyGlass v0.3 Xortron7MethedUpSwitchedUp
Darkc · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Tiance llama 2 7b chat finetune
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
IN
InternLM2
InternLM · RichardErkhov
Params19.9B
Context32K
Q4_K_M12.0 GB
GGUF
QW
Qwen2.5 3B Instruct
Qwen · dahara1
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
QW
Qwen2.5 7b Instruct Cpt v1.1
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
MI
dolphin-2.6-mistral-7b-dpo-orca
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
hydra project ChatHercules 2.5 Mistral 7B
Mistral · RichardErkhov
Params470M
Context—
Q4_K_M0.3 GB
GGUF
MI
Mistral 7B Base SFT RRHF
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AM
ambrosfitz TinyLlama 1.1B Chat yawp
Ambrosfitz · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
SM
SmolLM2 135M
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
Coding
CO
CodeRosa 70B AB1
Coderosa · altomek
Params69B
Context4K
Smallest14.5 GB
Coding
KU
Kukedlc Neural Krishna Multiverse 7b
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 8B Instruct Gradient 1048k
Llama · Monor
Params8.03B
Context1024K
Q4_K_M4.9 GB
GGUF
LL
Meta-Llama-3-8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen1.5-0.5B-Chat
Qwen · RichardErkhov
Params620M
Context32K
Q4_K_M0.4 GB
GGUF
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
PH
M1LL1X microsoft phi 3 5 mini trained abliterated QA 400 all modules merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama2_Biomed_Meditron
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
BL
Bloom 560m
Bloom · RichardErkhov
Params816M
Context1K
Q4_K_M0.6 GB
GGUF
LO
Local Novel LLM project Ninja v1 NSFW 128k
Local · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev mistral alpaca sft sample 4
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FI
FireRed Image Edit 1.0 comfy
Firered · Novice25
Params20.4B
Context—
Smallest7.5 GB
GGUF
LL
AIML GEEK mini prompt finetuned CodeLlama7b
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding

Other quants

Keep browsing