OM

omni json

OmniGGUF by erukudeUpdated 2025-12-18
Download on Hugging Face
Parameters45M
Context—
Layers—
Architecturellama
Smallest quant0.1 GB
Quick answer

omni json needs about 0.6 GB of memory at its main file (a 0.1 GB file) with an 8K context window, so it fits on a 4 GB GPU. This repository has 2 quants, from 0.1 GB to 0.2 GB.

OmniJSON – a deterministic JSON‑only assistant based on Llama3.2

OmniJSON is a fine‑tuned Llama3.2 model that always returns a single valid JSON object. It is designed for applications that require structured, parseable output for tasks such as math, coding, extraction, and planning.

Good at

  • Deterministic JSON output
  • Structured responses for apps
  • Reliable parsing

Limits

  • No vision support
  • Limited context size (8k tokens)
  • Only JSON output

Best for

  • API services needing JSON responses
  • Chatbots that parse structured data
  • Automation scripts that consume JSON

Who it’s for. Developers building applications that need consistent, machine‑readable output. The model is ideal for services that require a single JSON object for every request.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
—0.1 GB 0.6 GB
Fits Get file →
—0.2 GB 0.7 GB
Fits Get file →

The 0.09 GB file requires about 0.6 GB of VRAM at 8k context. The 0.19 GB file needs about 0.7 GB of VRAM. Choose the smaller file for 8 GB VRAM or less, and the larger file if you have 8 GB or more available.

What GPU do you need for omni json?

These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: the best quant that fits is the main file (0.2 GB file, about 0.7 GB in use).

Specifications

Architecture from the GGUF header

Architecturellama
Parameters45M

Source

Quantized byerukude
FamilyOmni
Licenseapache-2.0
Downloads (HF)2,148
Last updated2025-12-18

Frequently asked questions about omni json

How much VRAM does omni json need?

About 0.6 GB at its main file with an 8K context window. That is the 0.1 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run omni json on an 8 GB GPU?

Yes, with the smaller files: the 0.2 GB file needs about 0.7 GB at 8K context. Larger quants in this repository do not fit in 8 GB.

What is the file size of omni json GGUF?

It depends on the quant: file is 0.1 GB, file is 0.2 GB.

Where can I download omni json GGUF?

The files are on Hugging Face in the erukude/omni-json repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

What license does omni json use?

Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.

Can I run OmniJSON on 8 GB VRAM?

Yes, the 0.09 GB quantized file uses about 0.6 GB of VRAM at 8k context, which fits on 8 GB GPUs.

Does OmniJSON support vision?

No, the model does not have vision capabilities.

Similar models

Browse all →

More from erukude

All by erukude →

More Omni models

All Omni →

Other tiny models

All tiny models →