slim summary phi 3 needs about 6.1 GB of memory at its main file (a 2.4 GB file) with an 8K context window, so it fits on a 8 GB GPU.
slim-summary-phi-3: a small, fast summarization model
slim-summary-phi-3 is a finetune of phi-3 mini (3.8B parameters) for function‑calling summarization. It is packaged as a 4_K_M quantized GGUF, providing high‑quality summarizations of complex business documents on a small, specialized locally‑deployable model.
Good at
- Summarizing complex business documents
- Generating structured python list summaries
- Function‑calling summarization
Limits
- No vision support
- Requires 8K context
- License is apache-2.0
Best for
- Local summarization of business reports
- Generating key‑point lists
- Deploying on low‑resource machines
Who it’s for. Developers needing a lightweight summarization model for local deployment. The model is suitable for applications that require structured summaries and function‑calling capabilities without large GPU memory.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| — | 2.4 GB | 4.5 GB | Fits | Get file → |
The only available quant is 4_K_M with a file size of 2.39 GB. It requires approximately 6.1 GB of VRAM at 8K context. For 8 GB VRAM, this quant is suitable. If more memory is available, no other quants are provided.
What GPU do you need for slim summary phi 3?
These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: no quant fits at an 8K context.
- 6 GB VRAM: no quant fits at an 8K context.
- 8 GB VRAM: the best quant that fits is the main file (2.4 GB file, about 6.1 GB in use).
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about slim summary phi 3
How much VRAM does slim summary phi 3 need?
About 6.1 GB at its main file with an 8K context window. That is the 2.4 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run slim summary phi 3 on an 8 GB GPU?
Yes, with the smaller files: the 2.4 GB file needs about 6.1 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of slim summary phi 3 GGUF?
It depends on the quant: file is 2.4 GB.
What context length does slim summary phi 3 support?
The GGUF header lists a trained context length of 4,096 tokens (4K). The KV cache grows with the context you use: about 3.2 GB at 8K and 1.6 GB at 4K.
Where can I download slim summary phi 3 GGUF?
The files are on Hugging Face in the llmware/slim-summary-phi-3-gguf repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does slim summary phi 3 use?
Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run slim-summary-phi-3 on 8 GB VRAM?
Yes, the 4_K_M quant requires about 6.1 GB of VRAM at 8K context, which fits within an 8 GB GPU.
Does slim-summary-phi-3 support vision tasks?
No, the model does not support vision capabilities.