Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
Llama-3-Instruct-8B-SPPO-Iter3
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
QW
Qwen2 Math 1.5B Instruct
Qwen · legraphista
Params1.54B
Context4K
Smallest0.4 GB
FL
PornMaster flux2 klein 9b
Flux · VenerableRolliPolli
Params8.19B
Context256K
Q4_K_M5.0 GB
LL
Llama 3 Meerkat 8b v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
LI
Lily-Cybersecurity-7B-v0.2
Lily · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
WA
wav2vec2 large xlsr 53 english
Wav · cstr
Params315M
Context—
Smallest0.2 GB
IN
InternLM2
InternLM · legraphista
Params1.89B
Context8K
Smallest0.5 GB
GL
glm-4-9b-chat
GLM · legraphista
Params9.4B
Context128K
Smallest3.1 GB
RW
rwkv 6 finch 14b
Rwkv · latestissue
Params14.1B
Context1024K
Smallest5.4 GB
YU
Yuan2 M32 hf IMat
Yuan · legraphista
Params40.2B
Context4K
Smallest8.7 GB
SW
Swallow MS 7b Instruct v0.1
Swallow · RichardErkhov
Params7.33B
Context4K
Q4_K_M4.4 GB
LL
Llama-3-8B-Instruct-MopeyMule
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
BR
Breeze 7B 32k Instruct V1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
OM
omnivoice-tokenizer
Omnivoice · cstr
Params613M
Context—
Smallest0.3 GB
LC
LCM SSD 1B
Lcm · OlegSkutte
Params2.23B
Context—
Smallest2.2 GB
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
Chinese Mixtral 8x7B
Mistral · RichardErkhov
Params46.9B
Context32K
Q4_K_M28.6 GB
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
IN
indextts 1.5
Indextts · cstr
Params134M
Context—
Smallest0.3 GB
GR
Granite 3b Code Instruct 128k
Granite · RichardErkhov
Params3.48B
Context125K
Q4_K_M2.1 GB
FA
Falcon H1 7B Instruct
Falcon · tiiuae
Params7.59B
Context256K
Q4_K_M4.6 GB
GE
gemma 4 31B it qat uncensored heretic UDmerge
Gemma · EZForever
Params30.7B
Context256K
Smallest17.3 GB
SN
SniffyOtter 7B
Sniffyotter · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
saraprice llama2 7B backdoor DEPLOYMENT
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
KO
Kolibri 1 APEX
Kolibri · Myric
Params78.1B
Context—
Smallest0.4 GB
MO
moss tts v1.5
Moss · cstr
Params1.77B
Context—
Smallest3.5 GB
MI
MiniMax-M3.1-Flash
Minimax · AtomicChat
Params426B
Context1024K
Smallest112.2 GB
FL
FLUX.2 klein 4B
Flux · adithya-balaji
Params3.88B
Context—
Smallest1.6 GB
DE
DeepSeek-Coder-V2-Instruct
DeepSeek · legraphista
Params236B
Context160K
Smallest47.4 GB
QV
QVikhr 3 4B Instruction
Qvikhr · Vikhrmodels
Params4B
Context40K
Q4_K_M2.5 GB