Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
QW
Rombos LLM V2.5 Qwen 32b
Qwen · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB
LL
ReBatch Llama 3 8B dutch
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
klcsp llama3 8b fft summarization 11 v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
mlfoundations dev llama2 oh teknium scaling down random 0.2
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
Mistral 7B Instruct RRHF
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
microsoft Phi 3 mini 4k instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
VI
Vizzy CA plates fine tuned gpt2 minimal test
Vizzy · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
MI
GraySwanAI Mistral 7B Instruct RR
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
OL
OLMoE 1B 7B 0924
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
SY
Synatra_TbST02M_IN01
Synatra · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
LL
Zhao Ching TWLLM Llama2 Extend VTLoss Enhanced
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
ME
MetaMath Cybertron
Metamath · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ME
MelangeB-70b
Melangeb · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
GE
jstredacted gemma 2b instruct trivia qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
HU
ai human lab EEVE Korean Instruct 10.8B expo
Human · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
LL
Gemma 2b
Llama · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
nsmagar llama 2 7b law korean
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
OP
Openchat_3.5 Gpt 4 80k
Openchat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
nbeerbower llama3.1 gutenberg 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
TU
Tulu 2 Dpo 13b
Tulu · RichardErkhov
Params13B
Context8K
Q4_K_M7.9 GB
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
SW
v000000 SwallowMaid 8B L3 SPPO abliterated
Swallowmaid · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
CA
Calme-7B-Instruct-v0.1.1
Calme · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
failspy Llama 3 8B Instruct abliterated
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
VI
it-5.3-fp16-32k
Vikhrmodels · Vikhrmodels
Params7.63B
Context32K
Q4_K_M4.6 GB
CA
calme-2.3-legalkit-8b
Calme · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Mistral Megamerge Dare 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AL
Alphacode AI AlphaMist7B slr v4 slow
Alphacode · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral 7B Instruct IPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SA
SanjiWatsuki neural chat 7b v3 3 wizardmath dare me
Sanjiwatsuki · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB