Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · Saviitt
Params1.86B
Context—
Smallest0.9 GB
GGUF
SO
SOLAR 10B Nector DPO Jawade
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
TI
TinyLlama-1.1B-intermediate-step-1431k-3T-laser-dpo
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
LL
LLaMAntino 2 Chat 13b Hf UltraChat ITA
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
QW
Qwen2-0.5B-OnlineDPO-GRM-Gemma
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
UN
universeTBD astrollama 7b chat alpha
Universetbd · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
CR
crestf411 L3.1 8B sunfall stheno v0.6.1
Crestf · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
NA
Nanbeige2 16B Chat
Nanbeige · RichardErkhov
Params15.8B
Context4K
Q4_K_M9.6 GB
GGUF
BR
BroAlanTaps GPT2 large 64 20000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
TA
Taiwan-LLM-7B-v2.0.1-chat
Taiwan · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
PR
PranavHarshan MedNarraT
Pranavharshan · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
GE
google_gemma-3-1b-it
Gemma · Mungert
Params1B
Context32K
Smallest0.7 GB
GGUF
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
VI
Vikhr 7B instruct 0.4
Vikhr · Vikhrmodels
Params7.63B
Context32K
Q4_K_M4.6 GB
GGUF
BL
bluuwhale L3 SthenoMaidBlackroot 8B V1
Bluuwhale · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
ihopper ko gemma 7b sft dpo v1.0
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
MI
MistralInstructLongish
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev hp ablations mistral epoch1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
AIML GEEK promptv2 finetuned CodeLlama7b Instruct
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
mlfoundations dev llama2 oh teknium scaling down random 0.1
Llama · RichardErkhov
Params7B
Context4K
Q4_K_M4.1 GB
GGUF
GE
Gemma 2 27b Chatml
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
ST
Stablelm 3b 4e1t
Stablelm · RichardErkhov
Params2.8B
Context4K
Q4_K_M1.7 GB
GGUF
MI
avemio digital Mistral 7b orpo 3000 steps Prune
Mistral · RichardErkhov
Params5.06B
Context32K
Q4_K_M3.0 GB
GGUF
GR
grok 1
Grok · RichardErkhov
Params316B
Context8K
Smallest65.5 GB
GGUF
LL
AndreyRzhaksinskiy CDS CodeLlama Ins 7b E2E 20241022
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
QW
Qwen2.5 0.5B Instruct
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
ZO
zomd AISquare Instruct yi ko 6b v0.9.30
Zomd · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
NO
nota ai st vicuna v1.3 5.5b taylor
Nota · RichardErkhov
Params5.52B
Context2K
Q4_K_M3.3 GB
GGUF
GE
Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
TE
TeenyTinyLlama-460m-Chat
Teenytinyllama · RichardErkhov
Params468M
Context2K
Q4_K_M0.3 GB
GGUF

Other quants

Keep browsing