omni json needs about 0.6 GB of memory at its main file (a 0.1 GB file) with an 8K context window, so it fits on a 4 GB GPU. This repository has 2 quants, from 0.1 GB to 0.2 GB.
OmniJSON – a deterministic JSON‑only assistant based on Llama3.2
OmniJSON is a fine‑tuned Llama3.2 model that always returns a single valid JSON object. It is designed for applications that require structured, parseable output for tasks such as math, coding, extraction, and planning.
Good at
- Deterministic JSON output
- Structured responses for apps
- Reliable parsing
Limits
- No vision support
- Limited context size (8k tokens)
- Only JSON output
Best for
- API services needing JSON responses
- Chatbots that parse structured data
- Automation scripts that consume JSON
Who it’s for. Developers building applications that need consistent, machine‑readable output. The model is ideal for services that require a single JSON object for every request.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| — | 0.1 GB | 0.6 GB | Fits | Get file → | |
| — | 0.2 GB | 0.7 GB | Fits | Get file → |
The 0.09 GB file requires about 0.6 GB of VRAM at 8k context. The 0.19 GB file needs about 0.7 GB of VRAM. Choose the smaller file for 8 GB VRAM or less, and the larger file if you have 8 GB or more available.
What GPU do you need for omni json?
These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: the best quant that fits is the main file (0.2 GB file, about 0.7 GB in use).
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about omni json
How much VRAM does omni json need?
About 0.6 GB at its main file with an 8K context window. That is the 0.1 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run omni json on an 8 GB GPU?
Yes, with the smaller files: the 0.2 GB file needs about 0.7 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of omni json GGUF?
It depends on the quant: file is 0.1 GB, file is 0.2 GB.
Where can I download omni json GGUF?
The files are on Hugging Face in the erukude/omni-json repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does omni json use?
Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run OmniJSON on 8 GB VRAM?
Yes, the 0.09 GB quantized file uses about 0.6 GB of VRAM at 8k context, which fits on 8 GB GPUs.
Does OmniJSON support vision?
No, the model does not have vision capabilities.