CH

Chinese Decision 0.8b

ChineseGGUF by quincyqiangUpdated 2026-10-18
Download on Hugging Face
Parameters752M
Context256K
Layers24
Architectureqwen35
Smallest quant0.8 GB
Quick answer

Chinese Decision 0.8b needs about 1.7 GB of memory at Q8_0 (a 0.8 GB file) with an 8K context window, so it fits on a 4 GB GPU. Longer context needs more: at 32K tokens the KV cache alone adds about 1.6 GB.

Chinese Decision 0.8B – a lightweight Chinese decision model

Chinese Decision 0.8B is a lightweight decision model fine‑tuned from Qwen3.5‑0.8B. It returns probabilities for a fixed set of candidates without generating tokens. The model supports yes/no, single‑choice, and ordinal questions in general, medical, legal, and finance domains.

Good at

  • Fast single‑forward probability prediction
  • No token generation, zero output
  • Supports multiple Chinese domains

Limits

  • No vision or chat capabilities
  • Requires fixed candidate set
  • Context length limited to 262144 tokens

Best for

  • Intent recognition and routing
  • Content moderation and safety checks
  • Real‑time live‑stream comment filtering

Who it’s for. Engineers building decision‑making pipelines that need quick, low‑latency probability outputs for a limited set of options. Ideal for backend services that integrate with rule engines or human‑in‑the‑loop workflows.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
Q8_00.8 GB 1.7 GB
Fits Get file →

The Q8_0 GGUF file is 0.81 GB and requires approximately 1.7 GB of VRAM at an 8 k context. For CPU deployment, the same file can be loaded with llama‑cpp‑python. If memory is constrained, consider using the BF16 safetensors version (1.5 GB) with a GPU that supports FP16.

What GPU do you need for Chinese Decision 0.8b?

These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: the best quant that fits is Q8_0 (0.8 GB file, about 1.7 GB in use).

Which Chinese Decision 0.8b quant should you choose?

Q8_0 (0.8 GB) is the middle option in this repository. Close to lossless. It costs roughly twice the memory of a 4-bit file, so it suits cases where you have plenty of VRAM.

New to quant names? Read GGUF quantization explained.

Specifications

Architecture from the GGUF header

Architectureqwen35
Parameters752M
Layers24
Trained context262,144 tokens
KV cache (fp16)48 KB per token

Source

Quantized byquincyqiang
FamilyChinese
Licenseapache-2.0
Downloads (HF)1,086
Last updated2026-10-18

Frequently asked questions about Chinese Decision 0.8b

How much VRAM does Chinese Decision 0.8b need?

About 1.7 GB at Q8_0 with an 8K context window. That is the 0.8 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run Chinese Decision 0.8b on an 8 GB GPU?

Yes, with Q8_0: the 0.8 GB file needs about 1.7 GB at 8K context. Larger quants in this repository do not fit in 8 GB.

What is the file size of Chinese Decision 0.8b GGUF?

It depends on the quant: Q8_0 is 0.8 GB.

What context length does Chinese Decision 0.8b support?

The GGUF header lists a trained context length of 262,144 tokens (256K). The KV cache grows with the context you use: about 0.4 GB at 8K and 1.6 GB at 32K.

Where can I download Chinese Decision 0.8b GGUF?

The files are on Hugging Face in the quincyqiang/chinese-decision-0.8b-gguf repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does Chinese Decision 0.8b use?

Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run Chinese Decision 0.8B on 8 GB VRAM?

Yes, the Q8_0 GGUF requires about 1.7 GB of VRAM at 8 k context, well within an 8 GB GPU.

Does the model support vision or chat?

No, it only returns probabilities for a fixed candidate set and does not generate text or handle images.

What context length does it support?

The model was trained with 262144 tokens and can handle that context length in inference.

Similar models

Browse all →

More from quincyqiang

All by quincyqiang →

More Chinese models

All Chinese →

Other tiny models

All tiny models →