Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
OL
OLMo 7B 0424 hf Quantized
OLMo · aifoundry-org
Params6.89B
Context4K
Smallest1.6 GB
GE
Gemma 2 9B It SPPO Iter3
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GU
GA Guard AIO
Guard · prithivMLmods
Params4.02B
Context250K
Q4_K_M0.4 GB
LL
LLaDA Image Turbo
Llada · fszontagh
Params6.54B
Context—
Smallest7.0 GB
HE
0 hero Matter 0.1 7B
Hero · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
HE
0 hero Matter 0.1 Slim 7B B
Hero · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
llm-compiler-13b
Llm · legraphista
Params13B
Context16K
Smallest2.9 GB
QW
Qwen3.5 4B Base
Qwen · titan0115
Params4.21B
Context256K
Q4_K_M2.7 GB
LL
BatsResearch Llama 3.1 8B bonito v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DE
deepseek-coder-6.7b-base
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GE
Gemma 7b
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
PL
Plamo 2 Translate
Plamo · mitmul
Params9.53B
Context10240K
Q4_K_M6.3 GB
YI
Yi-1.5-9B-32K
Yi · RichardErkhov
Params8.83B
Context32K
Q4_K_M5.3 GB
KU
kukulemon-spiked-9B
Kukulemon · RichardErkhov
Params8.99B
Context8K
Q4_K_M5.4 GB
MA
ms marco MiniLM L 12 v2
Marco · cstr
Params33M
Context512
Smallest0.0 GB
QW
Qwen2.5 7B Instruct Uncensored
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
Meta Llama 3 70B Instruct
Llama · hermes42
Params70.6B
Context8K
Q4_K_M42.5 GB
LL
vicgalle Humanish Roleplay Llama 3.1 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
FL
flux1 tests
Flux · TylorShine
Params11.9B
Context—
Q4_K_M6.9 GB
BA
baichuan2 7b chat
Baichuan · shaowenchen
Params7.51B
Context4K
Smallest3.3 GB
GE
Turkish Gemma 9b T1
Gemma · ytu-ce-cosmos
Params9.24B
Context8K
Q4_K_M5.8 GB
ZE
gguf zephyr 220m dpo full
Zephyr · Felladrin
Params218M
Context2K
Q4_K_M0.2 GB
LL
Llama 2 7b Ebft Sparsegpt Outlier Wiki Block Outlier
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GR
granite-8b-code-instruct
Granite · SanctumAI
Params8.05B
Context4K
Q4_K_M4.9 GB
LL
vicgalle Configurable Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2 7B Instruct Step DPO
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
Meta Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
LL
llama2-7b-chat-hf-v3
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
llm-compiler-13b-ftd
Llm · legraphista
Params13B
Context16K
Smallest2.9 GB
SE
SeaLLMs v3 1.5B Chat
Seallms · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB