Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
SE
SeaLLMs v3 1.5B
Seallms · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
MI
mixtral
Mistral · hfl
Params46.7B
Context32K
Smallest9.5 GB
WA
wav2vec2 large xlsr 53 spanish
Wav · cstr
Params350M
Context—
Smallest0.0 GB
QW
Qwen Image 2.1 Turbo Uncensored
Qwen · Riyan200324200324
Params8.19B
Context256K
Smallest3.3 GB
UM
v000000 L3 Umbral Storm 8B t0.0001
Umbral · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
ST
StructLM 7B
Structlm · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
Llama 3 Instruct 8B SimPO v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
CH
Chatntq Ja 7b v1.0
Chatntq · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
HI
higgs audio v3 stt
Higgs · cstr
Params2.68B
Context—
Smallest0.0 GB
LL
AIML GEEK prompt finetuned CodeLlama7b Instruct
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · Unrestricted
Params7.52B
Context128K
Q4_K_M5.3 GB
LL
cais HarmBench Llama 2 13b cls
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
JS
jstredacted gamma 2b instruct ft awesome chatgpt prompts
Jstredacted · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
RE
reazonspeech-nemo-v2
Reazonspeech · sakasegawa
Params619M
Context—
Smallest0.4 GB
YU
Yuma42 KangalKhan RawRuby 7B
Yuma · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
EM
emailgen-pythia-410m-deduped
Emailgen · RichardErkhov
Params405M
Context2K
Q4_K_M0.3 GB
PH
TroyDoesAI Phi 3 Context Obedient RAG
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
LL
Llama Phishsense 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
NO
nothingiisreal L3 8B Instruct Abliterated DWP
Nothingiisreal · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
Qwen2.5 72B
Qwen · RichardErkhov
Params72.7B
Context128K
Q4_K_M47.4 GB
LL
laislemke LLaMA 2 vicuna 7b slerp
Llama · RichardErkhov
Params6.74B
Context32K
Q4_K_M0.6 GB
LL
Evolved Llama3 ELYZA Uncensored 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
PY
pygmalion-1.3b
Pygmalion · RichardErkhov
Params1.41B
Context2K
Q4_K_M0.9 GB
LL
Aion RP Llama 3.1 8B
Llama · aion-labs
Params8.03B
Context128K
Smallest3.2 GB
YU
yuntian deng gpt2 explicit cot multiplication 20 digits
Yuntian · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
LL
DataPilot Llama3.1 ArrowSE v0.4
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DA
DataPilot ArrowPro 7B KUJIRA
Datapilot · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
LL
Llama3 8B Aifeifei 1.0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
Huihui-Qwen3.6-35B-A3B-abliterated
Qwen · dnnagy
Params35.5B
Context256K
Smallest21.3 GB