Llama 2 7b chat hf function calling v2 needs about 0.5 GB of memory at its main file (a 0.0 GB file) with an 8K context window, so it fits on a 4 GB GPU. This repository has 2 quants, from 0.0 GB to 4.1 GB (Q3_K_M).
Llama 2 7b chat with function calling v2
Llama 2 7b chat hf function calling v2 is a 7B parameter model that supports structured JSON function calls. It is optimized for function calling with minimal syntax. The model is available in GGUF format with Q3_K_M quantization.
Good at
- Function calling with JSON output
- Low memory footprint with Q3_K_M
- Compact 7B parameter size
Limits
- No vision support
- Context length limited to 8k tokens
- License terms not specified
Best for
- API integration requiring function calls
- Lightweight chatbot deployments
- Rapid prototyping of function calling workflows
Who it’s for. Developers building chat interfaces that need reliable function calling. Researchers testing function calling mechanics in a 7B model.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| — | 0.0 GB | 0.5 GB | Fits | Get file → | |
| Q3_K_M | 4.1 GB | 5.1 GB | Fits | Get file → |
For 8 GB VRAM, use the Q3_K_M quantized file (4.08 GB). It requires about 5.1 GB of VRAM at 8k context. The unknown quant file is 0 GB but memory estimate is 0.5 GB, suitable for very low‑budget setups.
What GPU do you need for Llama 2 7b chat hf function calling v2?
These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: the best quant that fits is the main file (0.0 GB file, about 0.5 GB in use).
- 6 GB VRAM: the best quant that fits is Q3_K_M (4.1 GB file, about 5.1 GB in use).
Specifications
Architecture from the GGUF header
Frequently asked questions about Llama 2 7b chat hf function calling v2
How much VRAM does Llama 2 7b chat hf function calling v2 need?
About 0.5 GB at its main file with an 8K context window. That is the 0.0 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run Llama 2 7b chat hf function calling v2 on an 8 GB GPU?
Yes, with Q3_K_M: the 4.1 GB file needs about 5.1 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of Llama 2 7b chat hf function calling v2 GGUF?
It depends on the quant: file is 0.0 GB, Q3_K_M is 4.1 GB.
Where can I download Llama 2 7b chat hf function calling v2 GGUF?
The files are on Hugging Face in the Trelis/Llama-2-7b-chat-hf-function-calling-v2 repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
Can I run this model on 8 GB VRAM?
Yes, the Q3_K_M quantized file uses about 5.1 GB of VRAM at 8k context, fitting within an 8 GB GPU.
Does this model support vision?
No, the model has no vision capabilities.
What is the context length?
The model is trained for 8k token context length.