Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
LDCC Instruct Llama 2 Ko 13B v4.2.8
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
PH
third intellect Phi 3 mini 4k instruct orca math word problems 200k model 16bit
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
LL
Ko En Llama2 13b Aligned
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
LL
mlfoundations dev llama3 1 8b oh v3.1 wo airoboros
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
douy parrot llama 2 13B epoch3
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
KU
Kukedlc Neural Krishna Multiverse 7b v2
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
NE
NeuralPipe 7B Slerp
Neuralpipe · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · cvgro
Params1.86B
Context256K
Smallest0.9 GB
XL
xLAM 7b R
Xlam · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RH
rho-math-7b-interpreter-v0.1
Rho · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SY
SystemGemma2 2b It
Systemgemma · RichardErkhov
Params2.61B
Context32K
Q4_K_M1.7 GB
DE
deepseek-coder-1.3b-instruct
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
LL
Llama 2 7b chat_FFT_CodeAlpaca 20k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
genloop medical insurance llama 3.1 8B instruct merged
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GI
giraffe176 Starling Monarch Westlake Garten 7B v0.1
Giraffe · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Bitnet Mistral.0.2 47m v0.2 Grokfast1
Mistral · RichardErkhov
Params330M
Context4K
Q4_K_M0.2 GB
ST
Starcoderbase 1b
StarCoder · RichardErkhov
Params1.24B
Context8K
Q4_K_M0.8 GB
QW
Agnuxo Qwen2 1.5B Instruct MOE CODE assistant 16bit
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
QW
Qwen2.5 32B Instruct
Qwen · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.8 GB
JP
jpalgo MARK 3000 QLORA v2
Jpalgo · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
BT
bttr-handwritten-math-ocr
Bttr · cstr
Params7M
Context—
Smallest0.0 GB
OL
OLMo 7B 0724 Instruct Hf
OLMo · RichardErkhov
Params6.89B
Context2K
Q4_K_M4.2 GB
LL
Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GE
GeneZC MiniChat 1.5 3B
Genezc · RichardErkhov
Params3.02B
Context4K
Q4_K_M1.8 GB
LL
codellama CodeLlama 34b hf
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
OP
openchat-3.5-0106-11b
Openchat · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
LL
Llama 3 Youko 70b Instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
SY
SystemGemma2 9b It
Systemgemma · RichardErkhov
Params9.24B
Context32K
Q4_K_M5.8 GB
EN
ennodata-13b-8bit-raw-15epoch
Ennodata · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB