Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
CH
chinese alpaca 2 13b 16k
Chinese · hfl
Params13.3B
Context16K
Smallest5.0 GB
DE
deepseek coder 7b instruct
DeepSeek · jtrancas
Params6.74B
Context16K
Smallest2.8 GB
TI
TIPO 500M
Tipo · RichardErkhov
Params508M
Context2K
Q4_K_M0.3 GB
MI
nbeerbower Stella mistral nemo 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
PA
parakeet ctc 0.6b
Parakeet · cstr
Params609M
Context—
Smallest0.4 GB
GE
gemma-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
IN
AlchemistCoder L 7B
InternLM · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
ST
SD3.5 Large Turbo GGUF mixed sdcpp
Stable · stduhpf
Params8.23B
Context—
Smallest4.7 GB
LL
Llama 3 SynE
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
nbeerbower mistral nemo wissenschaft 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
ST
stt it fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
SR
SRPO
Srpo · befox
Params11.9B
Context—
Smallest4.0 GB
ST
stt fa fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
MI
Ae Calem Mistral 7b v0.1
Mistral · RichardErkhov
Params7.29B
Context32K
Q4_K_M4.4 GB
GA
garage bAInd Stable Platypus2 13B
Garage · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MI
Vezora Mistral 22B v0.2
Mistral · RichardErkhov
Params22.2B
Context64K
Q4_K_M13.3 GB
GE
Gemma 2b Aps It
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
EM
embeddinggemma 300m GGUF with dense modules
Embeddinggemma · cduk
Params308M
Context2K
Smallest0.2 GB
GE
zelk12 recoilme gemma 2 Ataraxy 9B v0.1 t0.25
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
NE
nenad1002 quantum research bot v1.0
Nenad · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LO
LongAlpaca 70B
Longalpaca · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
KA
kartoffelbox turbo
Kartoffelbox · cstr
Params427M
Context—
Smallest0.5 GB
HE
0 hero Matter 0.1 Slim 7B A
Hero · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
OL
Olmo 3 7B Think
OLMo · eaddario
Params7.3B
Context64K
Q4_K_M4.5 GB
GI
giux78 zefiro 7b beta ITA v0.1
Giux · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
stt be fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
ST
stt hy fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
LF
LFM2.5-24B-A2B
Lfm · Arki05
Params24B
Context128K
Smallest0.0 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
MI
WikiHow Mistral Instruct 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB