LL

Llama3Ternary 10b

LlamaGGUF by CMSManhattanUpdated 2026-10-28
Download on Hugging Face
Parameters9.64B
Context8K
Layers32
Architecturellama
Smallest quant4.3 GB
Quick answer

Llama3Ternary 10b needs about 5.9 GB of memory at TQ2_0 (a 4.3 GB file) with an 8K context window, so it fits on a 6 GB GPU. This repository has 2 quants, from 4.3 GB (TQ2_0) to 11.2 GB (Q8_0).

Llama3Ternary 10b – a ternary‑quantized 10B Llama 3 model for low‑RAM chat

Llama3Ternary 10b is a ternary (1.58‑bit) derivative of Meta‑Llama‑3‑8B with an expanded FFN, giving about 9.64 billion parameters. It is quantized to {-1,0,1} and designed for Microsoft BitNet.cpp kernels. The model supports a context length of 8192 tokens.

Good at

  • Low‑RAM inference with ternary quantization
  • Fast inference on AVX2/AVX‑512 CPUs
  • Chat‑ready 10B parameter size

Limits

  • Requires BitNet.cpp, not stock llama.cpp
  • No vision or multimodal support
  • License is llama3 community license

Best for

  • Home or small‑company chat bots
  • Low‑cost domain adaptation with LoRa
  • Fast inference on commodity CPUs

Who it’s for. Developers and small teams needing a 10B chat model that fits in limited RAM and runs quickly on standard CPUs. The model is suitable for users who can use Microsoft BitNet.cpp and are comfortable with a ternary quantized format.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
TQ2_04.3 GB 5.9 GB
Fits Get file →
Q8_011.2 GB 12.8 GB
Too big Get file →

For 8 GB VRAM, use the TQ2_0 quantization (file size 4.32 GB, estimated memory 5.9 GB at 8k context). For higher memory budgets, Q8_0 (11.24 GB, 12.8 GB at 8k context) offers full precision. Estimates are approximate.

What GPU do you need for Llama3Ternary 10b?

These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: no quant fits at an 8K context.
  • 6 GB VRAM: the best quant that fits is TQ2_0 (4.3 GB file, about 5.9 GB in use).
  • 8 GB VRAM: the best quant that fits is TQ2_0 (4.3 GB file, about 5.9 GB in use).
  • 12 GB VRAM: the best quant that fits is TQ2_0 (4.3 GB file, about 5.9 GB in use).
  • 16 GB VRAM: the best quant that fits is Q8_0 (11.2 GB file, about 12.8 GB in use).

Which Llama3Ternary 10b quant should you choose?

TQ2_0 (4.3 GB) is the middle option in this repository. Lower bit counts mean smaller files and more quality loss.

New to quant names? Read GGUF quantization explained.

Specifications

Architecture from the GGUF header

Architecturellama
Parameters9.64B
Layers32
Trained context8,192 tokens
KV cache (fp16)128 KB per token

Source

Quantized byCMSManhattan
FamilyLlama
Licensellama3
Downloads (HF)3,456
Last updated2026-10-28

Frequently asked questions about Llama3Ternary 10b

How much VRAM does Llama3Ternary 10b need?

About 5.9 GB at TQ2_0 with an 8K context window. That is the 4.3 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run Llama3Ternary 10b on an 8 GB GPU?

Yes, with TQ2_0: the 4.3 GB file needs about 5.9 GB at 8K context. Larger quants in this repository do not fit in 8 GB.

What is the file size of Llama3Ternary 10b GGUF?

It depends on the quant: TQ2_0 is 4.3 GB, Q8_0 is 11.2 GB.

What context length does Llama3Ternary 10b support?

The GGUF header lists a trained context length of 8,192 tokens (8K). The KV cache grows with the context you use: about 1.1 GB at 8K and 1.1 GB at 8K.

Where can I download Llama3Ternary 10b GGUF?

The files are on Hugging Face in the CMSManhattan/Llama3Ternary_10b repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does Llama3Ternary 10b use?

Hugging Face lists the license as llama3. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run Llama3Ternary 10b on 8 GB VRAM?

Yes, the TQ2_0 quantization requires about 5.9 GB of VRAM at 8k context, fitting within an 8 GB budget.

Does this model support vision or multimodal inputs?

No, the model is purely text and does not include vision capabilities.

What inference engine is needed?

It requires Microsoft BitNet.cpp kernels (AVX2/AVX‑512); it will not load in stock llama.cpp.

Other GGUFs of Meta-Llama-3-8B

Similar models

Browse all →

More from CMSManhattan

All by CMSManhattan →

More Llama models

All Llama →

Other medium models

All medium models →