gliner2 base v1 needs about 0.5 GB of memory at Q4_K (a 0.0 GB file) with an 8K context window, so it fits on a 4 GB GPU. This repository has 2 quants, from 0.0 GB (Q4_K) to 0.1 GB (Q4_K).
GLiNER2 base v1 – 205M parameter multi‑task NER, classification, structured extraction, and relation extraction
GLiNER2 base v1 is a 205M parameter model that performs named entity recognition, text classification, structured data extraction, and relation extraction in a single forward pass. It is optimized for CPU inference and does not require a GPU. The model is distributed in GGUF format with Q4_K quantization.
Good at
- Unified multi‑task extraction
- Fast CPU inference
- Low‑latency local processing
Limits
- No GPU required but limited to 8k context
- No vision or audio capabilities
- License is Apache 2.0
Best for
- Local NER pipelines
- Text classification on edge devices
- Structured data extraction from documents
Who it’s for. Developers building local NLP pipelines that need multiple extraction tasks without external APIs. Researchers requiring a lightweight, multi‑task model for on‑device inference.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| Q4_K | 0.0 GB | 0.5 GB | Fits | Get file → | |
| Q4_K | 0.1 GB | 0.6 GB | Fits | Get file → |
For 8 GB VRAM, the Q4_K variant with file size 0.03 GB uses approximately 0.5 GB memory at 8k context. If more memory is available, the 0.11 GB Q4_K variant uses about 0.6 GB. Both variants are suitable for CPU inference and meet the 8k context limit.
What GPU do you need for gliner2 base v1?
These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: the best quant that fits is Q4_K (0.1 GB file, about 0.6 GB in use).
Which gliner2 base v1 quant should you choose?
Q4_K (0.0 GB) is the middle option in this repository. The usual sweet spot between file size and quality, and the quant most people start with.
New to quant names? Read GGUF quantization explained.
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about gliner2 base v1
How much VRAM does gliner2 base v1 need?
About 0.5 GB at Q4_K with an 8K context window. That is the 0.0 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run gliner2 base v1 on an 8 GB GPU?
Yes, with Q4_K: the 0.1 GB file needs about 0.6 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of gliner2 base v1 GGUF?
It depends on the quant: Q4_K is 0.0 GB, Q4_K is 0.1 GB.
Where can I download gliner2 base v1 GGUF?
The files are on Hugging Face in the antflydb/gliner2-base-v1 repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
Can I run GLiNER2 base v1 on 8 GB VRAM?
Yes, the Q4_K GGUF files use less than 1 GB of VRAM, making them suitable for 8 GB VRAM systems.
Does this model support vision tasks?
No, GLiNER2 base v1 is a text‑only model and does not include vision capabilities.
What license applies to this model?
The model is licensed under the Apache License 2.0.