Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
Tool use
LL
Llama-3-8B-Lexi-Uncensored
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SW
Swallow MX 8x7b NVE v0.1
Swallow · RichardErkhov
Params46.7B
Context4K
Q4_K_M28.4 GB
GGUF
EP
epiCRealism SD15
Epicrealism · ygohel18
Params1.07B
Context—
Smallest1.6 GB
GGUF
SW
Swallow 7b Hf
Swallow · RichardErkhov
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
VI
vicuna 13b v1.5 16k
Vicuna · shaowenchen
Params13B
Context16K
Smallest5.4 GB
GGUF
OL
OLMoE 1B 7B 0924 Instruct
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
GGUF
ST
Starcoder2 15b Fp16
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
Coding
GL
lid-fasttext-glotlid-v3
Glotlid · cstr
Params419M
Context—
Smallest0.2 GB
GGUF
PY
pythia_2.8b_deduped-task065_timetravel_consistent_sentence_classification
Pythia · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
PH
microsoft Phi 3 medium 128k instruct
Phi · RichardErkhov
Params14B
Context128K
Q4_K_M8.6 GB
GGUF
LL
jc9080 llama3 open ko 8b Instruct shimshimi 500 ver2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama PLLuM 70B Chat
Llama · Nondzu
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
BI
BioMistral BioMistral 7B DARE
Biomistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
QW
Canum med Qwen3 Reasoning
Qwen · prithivMLmods
Params1.72B
Context40K
Q4_K_M1.1 GB
ReasoningTool use
LL
Llama3.2 1b Train Aware Quantization
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
NO
nota ai st vicuna v1.3 5.5b ppl
Nota · RichardErkhov
Params5.52B
Context2K
Q4_K_M3.3 GB
GGUF
TE
Tensoic TinyLlama 1.1B 2.5T openhermes
Tensoic · RichardErkhov
Params1.1B
Context4K
Q4_K_M0.7 GB
Tool use
OR
orpheus_3b_0.1_GGUF
Orpheus · Prince-1
Params3.3B
Context128K
Q4_K_M2.1 GB
Tool use
GR
granite 4.1 8b
Granite · eaddario
Params8.79B
Context128K
Smallest0.0 GB
GGUF
PR
prometheus eval prometheus bgb 8x7b v2.0
Prometheus · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
WH
whisper large v3
Whisper · oxide-lab
Params1.61B
Context—
Smallest0.5 GB
GGUF
PH
Agnuxo Phi 3.5 Trinitron Instruct CODE Python English Asistant 16bit v2
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
Coding
OL
OLMoE 1B 7B 0924 Base
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
AR
ArmurAI Pentest AI
Armurai · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SE
seyf1elislam WestKunai Hermes 7b
Seyf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
CL
Climategpt 7b Fsg
Climategpt · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B Instruct ORPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing