Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
Llama-3-8B-grammar-correction
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
LL
ORLM LLaMA 3 8B
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen2.5 7B Instruct Abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
K2S3 Mistral 7b v1.4
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
IC
ichigoberry MonarchPipe 7B slerp
Ichigoberry · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
BR
Breeze TTS 2
Breeze · Raxdiam
Params—
Context—
Smallest4.3 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
flammenai flammen18 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
granite speech 4.1 2b plus
Granite · cstr
Params2.32B
Context—
Smallest1.7 GB
ML
mlfoundations dev stackoverflow 5000tasks 0p
Mlfoundations · RichardErkhov
Params8B
Context128K
Q4_K_M4.9 GB
GE
Gemma 2 2b It
Gemma · legraphista
Params2.61B
Context8K
Smallest0.8 GB
MA
MaidStarling 2x7B Base
Maidstarling · RichardErkhov
Params12.9B
Context8K
Q4_K_M7.8 GB
OP
OpenELM 3B Instruct
Openelm · RichardErkhov
Params3.04B
Context2K
Q4_K_M1.9 GB
CO
Yi Coder 1.5B Chat
Coder · RichardErkhov
Params1.48B
Context128K
Q4_K_M1.0 GB
LL
ZhangShenao SELM Llama 3 8B Instruct iter 2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
lemon mint gemma ko 7b it v0.40
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Tetra04
Params34.7B
Context256K
Q4_K_M21.2 GB
VI
Vinallama 7b Chat
Vinallama · RichardErkhov
Params6.86B
Context4K
Q4_K_M4.2 GB
MI
Mistral 7B v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
NX
Nxcode-CQ-7B-orpo
Nxcode · Monor
Params7.25B
Context64K
Q4_K_M4.7 GB
LL
MrRobotoAI llama3 8B Special Dark v3.1.2c
Llama · RichardErkhov
Params8B
Context1024K
Q4_K_M4.9 GB
GR
granite-3b-code-instruct
Granite · SanctumAI
Params3.48B
Context2K
Q4_K_M2.1 GB
QW
Qwen3.8 27B
Qwen · eaddario
Params27.3B
Context—
Smallest0.0 GB
ST
Kyutai-STT
Stt · cstr
Params1.05B
Context—
Smallest0.7 GB
MA
Magnum Instruct DPO
Magnum · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
PH
Phi 3 mini 128k instruct IMat
Phi · legraphista
Params3.82B
Context128K
Smallest0.8 GB
LL
Llama 3 8B Instruct
Llama · hermes42
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen Image 2.1 Turbo Uncensored
Qwen · Zyszztggd
Params8.19B
Context—
Smallest3.3 GB
MO
moss tts local v1.5
Moss · cstr
Params2.12B
Context—
Smallest2.1 GB