LA

laya_typed_decisions_unsloth_dynamic

LayaGGUF by mysUpdated 2026-09-20
Download on Hugging Face
Parameters423M
Context—
Layers—
Architectureggmlc
Smallest quant0.4 GB
Quick answer

laya_typed_decisions_unsloth_dynamic needs about 1.0 GB of memory at Q4_K_M (a 0.4 GB file) with an 8K context window, so it fits on a 4 GB GPU. This repository has 3 quants, from 0.4 GB (Q4_K_M) to 0.8 GB (F16).

Specialist Laya model for typed decision workflows

Laya Typed-Decisions is a 0.423B parameter model fine‑tuned on four synthetic workflows: invoice match, SOC alerts, customer‑service next action, and agent‑trace harnesses. It outputs calibrated probabilities in a single encoder pass and is compiled with ggmlc for high‑performance GGML execution.

Good at

  • Typed decision scoring
  • One‑pass probability output
  • Workflow‑specific accuracy

Limits

  • Not autoregressive
  • Not compatible with llama.cpp
  • Limited to four synthetic workflows

Best for

  • Invoice matching
  • SOC alert triage
  • Customer‑service next action

Who it’s for. Developers building decision‑making services that require fast, calibrated probability outputs for specific typed‑question workflows. The model is ideal for backend services that need to evaluate many states in parallel.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
Q4_K_M0.4 GB 1.0 GB
Fits Get file →
Q8_00.5 GB 1.0 GB
Fits Get file →
F160.8 GB 1.5 GB
Fits Get file →

For 8 GB VRAM, use Q4_K_M (0.42 GB, ~1 GB memory at 8k context). For 16 GB VRAM, Q8_0 (0.46 GB, ~1 GB memory). If more precision is needed and memory allows, F16 (0.85 GB, ~1.5 GB memory).

What GPU do you need for laya_typed_decisions_unsloth_dynamic?

These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: the best quant that fits is F16 (0.8 GB file, about 1.5 GB in use).

Which laya_typed_decisions_unsloth_dynamic quant should you choose?

Q4_K_M is the usual starting point for laya_typed_decisions_unsloth_dynamic: it balances file size and quality well. Here it is a 0.4 GB file.

The Q8_0 file is 0.5 GB, about 1.1 times the size of Q4_K_M. It keeps the weights closer to the original, which only matters if you have the memory to spare.

New to quant names? Read GGUF quantization explained.

Specifications

Architecture from the GGUF header

Architectureggmlc
Parameters423M

Source

Quantized bymys
FamilyLaya
Licenseapache-2.0
Downloads (HF)742
Last updated2026-09-20

Frequently asked questions about laya_typed_decisions_unsloth_dynamic

How much VRAM does laya_typed_decisions_unsloth_dynamic need?

About 1.0 GB at Q4_K_M with an 8K context window. That is the 0.4 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run laya_typed_decisions_unsloth_dynamic on an 8 GB GPU?

Yes, with F16: the 0.8 GB file needs about 1.5 GB at 8K context. Larger quants in this repository do not fit in 8 GB.

What is the file size of laya_typed_decisions_unsloth_dynamic GGUF?

It depends on the quant: Q4_K_M is 0.4 GB, Q8_0 is 0.5 GB, F16 is 0.8 GB.

Where can I download laya_typed_decisions_unsloth_dynamic GGUF?

The files are on Hugging Face in the mys/laya-typed-decisions-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does laya_typed_decisions_unsloth_dynamic use?

Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run this model on 8 GB VRAM?

Yes, the Q4_K_M quantized file is 0.42 GB and requires about 1 GB of VRAM at 8k context.

Is this model compatible with llama.cpp?

No, the files are compiled with ggmlc and will not load in llama.cpp.

What workflows is this model specialized for?

Invoice match, SOC alerts, customer‑service next action, and agent‑trace harnesses.

Other GGUFs of laya-typed-decisions

Similar models

Browse all →

More from mys

All by mys →

More Laya models

All Laya →

Other tiny models

All tiny models →