Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
CY
z image cyberrealistic v8
Cyberrealistic · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
MI
Ministral 3 14B Instruct 2512
Mistral · eaddario
Params13.5B
Context256K
Q4_K_M8.2 GB
LL
Meta-Llama-3-8B-Instruct
Llama · SanctumAI
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
gemma 4 31b it assistant mtp
Gemma · NotMe404
Params470M
Context256K
Q4_K_M0.4 GB
MM
mmq test
Mmq · Beinsezii
Params14.8B
Context128K
Smallest0.4 GB
LL
Reflection Llama 3.1 70B IMat
Llama · legraphista
Params70.6B
Context8K
Smallest15.3 GB
MI
mistral-7b-v0.1-layla-v4-chatml
Mistral · l3utterfly
Params7.24B
Context32K
Q4_K_M4.4 GB
YI
Yi 1.5 6B
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
QW
Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive
Qwen · Unrestricted
Params122B
Context256K
Q4_K_M74.2 GB
GE
gemma 4 E4B it
Gemma · eaddario
Params7.52B
Context128K
Smallest0.0 GB
MO
moonshine-base
Moonshine · cstr
Params62M
Context—
Smallest0.0 GB
VA
VAGOsolutions SauerkrautLM 7b HerO
Vagosolutions · RichardErkhov
Params3.8B
Context—
Q4_K_M2.2 GB
LL
microsoft LLaMA 2 7b GTL Delta
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · ATOMIKMN
Params1.86B
Context—
Smallest0.9 GB
MI
Mixtral 8x7B Instruct v0.1
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
TY
Typhoon 7b Instruct 02 19 2024
Typhoon · RichardErkhov
Params7.27B
Context32K
Q4_K_M4.4 GB
MI
MistarlingMaid 2x7B Base
Mistarlingmaid · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
RA
raon speech 9b
Raon · cstr
Params8.55B
Context—
Smallest5.0 GB
QW
Qwen2.5 Coder 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
BR
Breeze TTS 2
Breeze · npario
Params—
Context—
Smallest4.3 GB
BR
Breeze 7B 32k Base V1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GL
Glm Edge V 5b
GLM · zai-org
Params4.33B
Context3K
Q4_K_M2.6 GB
LL
Hermes 3 Llama 3.1 70B
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
MI
nbeerbower Gutensuppe mistral nemo 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
KA
Karakuri Lm 7b Apm v0.1
Karakuri · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
TT
fastpitch en
Tts · cstr
Params60M
Context—
Smallest0.1 GB
GL
glm-4-9b-chat-1m
GLM · legraphista
Params9.48B
Context1024K
Smallest4.0 GB
MI
Mistral Large Instruct 2407
Mistral · legraphista
Params123B
Context32K
Smallest26.0 GB
GE
gemma 4 26B A4B it
Gemma · eaddario
Params25.2B
Context256K
Smallest0.1 GB