Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
IN
inflatebot MN 12B Mag Mell R1
Inflatebot · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Reasoning
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · khalifah66362666
Params1.86B
Context256K
Smallest0.9 GB
GGUF
XU
XueyingJia pythia 1b stage1 dpo SG merge
Xueyingjia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
RL
Rlhflow_Mixture_Iter2
Rlhflow · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
KaKee llama 2 7b full unknown various answer 7 stereo epoch1
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
TI
TinyLlama-1.1B-Chat-v0.2
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
hardikg2907 code llama html completion 1
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
BE
besimray miner id 3 72df7d62 e0d6 41b2 9153 9843320d6b82 1729802122
Besimray · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
QW
Qwen2 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
MI
K2S3 Mistral 7b v1.43
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
MI
WebraftAI synapsellm 7b mistral v0.4 preview3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ML
mlfoundations dev oh v1.3 alpaca x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LF
lfm2 colbert
Lfm · cstr
Params353M
Context—
Smallest0.0 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
XT
xTower13B
Xtower · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
Mistral 7B v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
TinyLlama TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Nous-Hermes-Llama2-13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
Tool use
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ME
MELT-TinyLlama-1.1B-Chat-v1.0
Melt · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
OP
OpenELM 1_1B DPO Full Max Min Reward
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
QW
Rombos LLM V2.5 Qwen 32b
Qwen · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB
Tool use
LL
ReBatch Llama 3 8B dutch
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
klcsp llama3 8b fft summarization 11 v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
mlfoundations dev llama2 oh teknium scaling down random 0.2
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B Instruct RRHF
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
microsoft Phi 3 mini 4k instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
VI
Vizzy CA plates fine tuned gpt2 minimal test
Vizzy · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
GraySwanAI Mistral 7B Instruct RR
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing