Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AL
Alphacode-MALI-11B_slowtest
Alphacode · RichardErkhov
Params10.9B
Context32K
Q4_K_M6.6 GB
IN
InnovationHacksAI tkgcore
Innovationhacksai · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
CO
CodeV DS 6.7B
Codev · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · yxccl
Params34.7B
Context256K
Q4_K_M21.2 GB
MI
CorticalStack mistral 7b openhermes 2.5 sft
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LE
Lens Turbo
Lens · rootonchair
Params4.1B
Context—
Smallest2.4 GB
CH
CharGen-v3-beta-rl-83-s0
Chargen · CharGen
Params23.6B
Context32K
Q4_K_M14.3 GB
GE
Gemma 2 9b
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
QW
Qwen2.5 14B Instruct
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
QW
Qwen2.5 1.5B Instruct
Qwen · dahara1
Params1.54B
Context32K
Q4_K_M1.0 GB
QW
Qwen2.5 14B Instruct
Qwen · dahara1
Params14.8B
Context32K
Q4_K_M9.0 GB
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · justfrfn
Params34.7B
Context256K
Q4_K_M21.2 GB
UN
Unbabel TowerInstruct 13B v0.1
Unbabel · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
LL
Meta Llama 3.1 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
mlfoundations dev hp ablations grid mistral bsz1024 lr2e 6 scheduler cosine warmup0.15
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
LION LLaMA 3 8b Dpo v1.0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
mlfoundations dev hp ablations mistral scheduler linear warmup0.10
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
bunnycore Phi 2 OmniMatrix
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
LL
codellama CodeLlama 34b Instruct hf
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
GE
Gemma 2 9b It 390K
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
LL
ihughes15234 llama 3 1 8bi tictactoe dpo1epoch v3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
kayfour T3Q llama3.1 8B it Ko safe
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
MH
MHGanainy gpt2 xl lora multi shared 512 top
Mhganainy · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
ME
Merdeka LLM merdeka llm lawyer 3b 128k instruct
Merdeka · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
OP
Opencsg Stable Code 3b v1
Opencsg · RichardErkhov
Params2.8B
Context16K
Q4_K_M1.7 GB
MA
martimfasantos tinyllama 1.1b mt sft full
Martimfasantos · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
Llama 3.2 3B Wildguard Ko 2410
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB