Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
DE
deepseek-coder-1.3b-chat-and-function-calling
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
ML
mlfoundations dev stackoverflow 5000tasks 1p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Meta-Llama-3-8B-Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LA
Labradorite 13b
Labradorite · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
VI
Vikhrmodels/Vikhr-YandexGPT-5-Lite-8B-it
Vikhrmodels · Vikhrmodels
Params8.04B
Context32K
Q4_K_M4.9 GB
QW
Replete LLM V2.5 Qwen 14b
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
TL
tlphams solar 10.7b merged v0.1
Tlphams · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
MI
Codestral-22B-v0.1
Mistral · SanctumAI
Params22.2B
Context32K
Q4_K_M13.3 GB
QW
Qwen3.5 9B
Qwen · titan0115
Params8.95B
Context256K
Q4_K_M5.6 GB
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
MA
MadKFC MergedCPsyModel
Madkfc · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
PA
parakeet tdt ctc 110m
Parakeet · cstr
Params115M
Context—
Smallest0.1 GB
FA
Falcon H1 Tiny Coder 90M
Falcon · tiiuae
Params108M
Context256K
Q4_K_M0.1 GB
NO
notHumpback-Myx
Nothumpback · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
LL
scandukuri llama3 8b stargate m1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GR
granite speech 4.0 1b
Granite · cstr
Params2.31B
Context—
Smallest2.9 GB
GE
Gemma 2b It
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
DI
distilgpt2
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
IN
InternLM2
InternLM · legraphista
Params7.74B
Context32K
Smallest1.9 GB
LE
Leia Swallow 7b
Leia · RichardErkhov
Params6.83B
Context4K
Q4_K_M4.1 GB
GL
GLM-5.3-Flash
GLM · neuralll
Params7.43B
Context1024K
Smallest4.6 GB
LL
uukuguy speechless codellama orca platypus 13b 0.10e
Llama · RichardErkhov
Params430M
Context—
Q4_K_M0.0 GB
TO
tomasonjo text2cypher demo 16bit
Tomasonjo · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
LL
ZhangShenao SELM Llama 3 8B Instruct iter 1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Llama 3.2 3B Instruct
Llama · SanctumAI
Params3.21B
Context128K
Q4_K_M2.0 GB
MI
Mistral Nemo Instruct 2407
Mistral · legraphista
Params12.2B
Context1000K
Smallest3.0 GB
GI
Gille StrangeMerges 52 7B dare ties
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
granite-7b-lab
Granite · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
NE
nemotron 3.5 asr streaming 0.6b
Nemotron · cstr
Params638M
Context—
Smallest0.4 GB
LL
LLaDA Image
Llada · fszontagh
Params6.54B
Context—
Smallest7.0 GB