NeoHorse Jev 4B needs about 6.1 GB of memory at Q8_0 (a 4.5 GB file) with an 8K context window, so it fits on a 8 GB GPU. Longer context needs more: at 32K tokens the KV cache alone adds about 4.3 GB.
NeoHorse Jev 4B – a 4B structured decision model for agent workflows
NeoHorse Jev 4B is a 4.21B parameter model built on the qwen35 architecture. It predicts application‑defined decisions and their probabilities without autoregressive text generation. The model supports prefill‑only inference and can be deployed with vLLM, SGLang, or the native runtime.
Good at
- Prefill‑only decision inference
- Application‑defined routing and tool selection
- High accuracy on structured decision tasks
Limits
- No autoregressive text generation
- Requires 6.1GB VRAM at 8k context
- Apache‑2.0 license only
Best for
- Agent workflow routing
- Tool selection logic
- Decision‑based automation
Who it’s for. Developers building local LLM agents that need fast, probability‑based decision making. The model is ideal for applications that require routing, tool selection, or rating decisions without generating full text responses.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| Q8_0 | 4.5 GB | 6.1 GB | Fits | Get file → |
The only available quant is Q8_0 with a file size of 4.48GB. It requires approximately 6.1GB of VRAM at an 8k context length. If your GPU has at least 8GB of VRAM, Q8_0 will run comfortably. For lower VRAM budgets, no other quant is available.
What GPU do you need for NeoHorse Jev 4B?
These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: no quant fits at an 8K context.
- 6 GB VRAM: no quant fits at an 8K context.
- 8 GB VRAM: the best quant that fits is Q8_0 (4.5 GB file, about 6.1 GB in use).
Which NeoHorse Jev 4B quant should you choose?
Q8_0 (4.5 GB) is the middle option in this repository. Close to lossless. It costs roughly twice the memory of a 4-bit file, so it suits cases where you have plenty of VRAM.
New to quant names? Read GGUF quantization explained.
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about NeoHorse Jev 4B
How much VRAM does NeoHorse Jev 4B need?
About 6.1 GB at Q8_0 with an 8K context window. That is the 4.5 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run NeoHorse Jev 4B on an 8 GB GPU?
Yes, with Q8_0: the 4.5 GB file needs about 6.1 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of NeoHorse Jev 4B GGUF?
It depends on the quant: Q8_0 is 4.5 GB.
What context length does NeoHorse Jev 4B support?
The GGUF header lists a trained context length of 262,144 tokens (256K). The KV cache grows with the context you use: about 1.1 GB at 8K and 4.3 GB at 32K.
Where can I download NeoHorse Jev 4B GGUF?
The files are on Hugging Face in the DreamBlooms/NeoHorse-Jev-4B-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does NeoHorse Jev 4B use?
Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run NeoHorse Jev 4B on 8 GB VRAM?
Yes, the Q8_0 quant requires about 6.1GB of VRAM at 8k context, so 8GB is sufficient.
Does NeoHorse Jev 4B support vision?
The model is primarily a decision model; image requests are optional and not core to its functionality.
What license is NeoHorse Jev 4B released under?
The model is released under the apache-2.0 license.