Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
QW
Qwen2-7B-Instruct
Qwen · legraphista
Params7.62B
Context32K
Smallest1.9 GB
OP
OpenCodeInterpreter-DS-1.3B
Opencodeinterpreter · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
RA
RakutenAI-7B-instruct
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
CH
chatterbox turbo
Chatterbox · cstr
Params266M
Context—
Smallest0.3 GB
CR
CREPE (tiny)
Crepe · cstr
Params22M
Context—
Smallest0.0 GB
SW
Swallow 70b Instruct v0.1
Swallow · RichardErkhov
Params69B
Context4K
Q4_K_M41.5 GB
TU
Tulu2 7b_writing Prompts
Tulu · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
rana shahroz mistral 7b openorca20k
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
IM
z image solordz v2
Image · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
GR
Granite 8b Code Base 4k
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
MI
Chinese Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.5B
Context32K
Q4_K_M4.5 GB
MI
voxtral mini 3b 2507
Mistral · cstr
Params4.68B
Context—
Smallest2.7 GB
CH
chujiezheng Starling LM 7B beta ExPO
Chujiezheng · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
Hermes 3 Llama 3.1 70B
Llama · legraphista
Params70.6B
Context128K
Smallest15.3 GB
CO
cohere transcribe arabic 07 2026
Cohere · cstr
Params2.07B
Context—
Smallest0.0 GB
RH
rho-math-1b-interpreter-v0.1
Rho · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
CA
canary 1b v2
Canary · cstr
Params980M
Context—
Smallest0.0 GB
MI
UCLA AGI Mistral7B PairRM SPPO Iter3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 32B Instruct Abliterated
Qwen · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB
ST
stabilityai japanese stablelm base gamma 7b
Stabilityai · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
LL
Llama 2 7b Chat Hf Gpt 4 80k Base_Lora
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
Llama3 8B Chinese Chat
Llama · Monor
Params8B
Context8K
Q4_K_M4.9 GB
GR
Granite 3b Code Base
Granite · RichardErkhov
Params3.48B
Context2K
Q4_K_M2.1 GB
NE
neo_7b_instruct_v0.1
Neo · legraphista
Params7.79B
Context8K
Smallest1.7 GB
QW
Qwen1.5-MoE-A2.7B-Chat
Qwen · RichardErkhov
Params14.3B
Context32K
Q4_K_M9.5 GB
AY
aya-23-8B
Aya · legraphista
Params8B
Context8K
Smallest2.2 GB
BE
berkeley nest Starling LM 7B alpha
Berkeley · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
Llama 3.2 1B Instruct
Llama · legraphista
Params1.24B
Context128K
Smallest0.4 GB
YI
01 ai Yi 9B 200K
Yi · RichardErkhov
Params8.83B
Context256K
Q4_K_M5.3 GB
QW
online-dpo-qwen2-4
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB