CO

CodeShell

CodeshellGGUF by WisdomShellUpdated 2023-11-08
Download on Hugging Face
Parameters7.98B
Context8K
Layers42
Architecturecodeshell
Smallest quant4.6 GB
Quick answer

CodeShell needs about 6.5 GB of memory at Q4_0 (a 4.6 GB file) with an 8K context window, so it fits on a 8 GB GPU.

CodeShell – a 7B multilingual code LLM with strong code generation performance

CodeShell is a 7.98 billion‑parameter code LLM developed by Peking University’s Knowledge Computing Lab and Sichuan Tianfu Bank AI team. It was trained on 500 billion tokens and offers an 8192‑token context window. The Int4 quantized version is available in GGUF format.

Good at

  • High‑quality code generation
  • Multi‑language coding support
  • Fast local inference with Int4 quantization

Limits

  • No vision capabilities
  • License not specified
  • Requires 8 k context for full performance

Best for

  • Developing code assistants
  • Generating test cases
  • Debugging code snippets

Who it’s for. Engineers and developers seeking a lightweight, local code generation model. The Int4 quantized version runs efficiently on modest hardware, making it suitable for on‑premise deployments and IDE integration.

Quants & memory

QuantFileEst. memoryvs VRAMResultDownload
Q4_04.6 GB 6.5 GB
Fits Get file →

The Q4_0 quantized file is 4.57 GB and requires approximately 6.5 GB of VRAM at 8 k context. This is suitable for systems with 8 GB or more VRAM. For lower VRAM budgets, a different quantization would be needed, but only Q4_0 is provided.

What GPU do you need for CodeShell?

These figures add the file size, an fp16 KV cache at an 8K context window (worked out from this model’s own header) and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.

  • 4 GB VRAM: no quant fits at an 8K context.
  • 6 GB VRAM: no quant fits at an 8K context.
  • 8 GB VRAM: the best quant that fits is Q4_0 (4.6 GB file, about 6.5 GB in use).

Which CodeShell quant should you choose?

Q4_0 (4.6 GB) is the middle option in this repository. The usual sweet spot between file size and quality, and the quant most people start with.

New to quant names? Read GGUF quantization explained.

Specifications

Architecture from the GGUF header

Architecturecodeshell
Parameters7.98B
Layers42
Trained context8,192 tokens
KV cache (fp16)168 KB per token

Source

Quantized byWisdomShell
FamilyCodeshell
Downloads (HF)1,126
Last updated2023-11-08

Frequently asked questions about CodeShell

How much VRAM does CodeShell need?

About 6.5 GB at Q4_0 with an 8K context window. That is the 4.6 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.

Can I run CodeShell on an 8 GB GPU?

Yes, with Q4_0: the 4.6 GB file needs about 6.5 GB at 8K context. Larger quants in this repository do not fit in 8 GB.

What is the file size of CodeShell GGUF?

It depends on the quant: Q4_0 is 4.6 GB.

What context length does CodeShell support?

The GGUF header lists a trained context length of 8,192 tokens (8K). The KV cache grows with the context you use: about 1.4 GB at 8K and 1.4 GB at 8K.

Where can I download CodeShell GGUF?

The files are on Hugging Face in the WisdomShell/CodeShell-7B-Chat-int4 repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.

Can I run CodeShell on 8 GB VRAM?

Yes, the Q4_0 quantized model requires about 6.5 GB of VRAM at 8 k context, fitting within an 8 GB GPU.

Does CodeShell support vision tasks?

No, CodeShell is a text‑only code model with no vision capabilities.

What context length does CodeShell use?

CodeShell supports a context window of 8192 tokens.

Similar models

Browse all →

More Codeshell models

All Codeshell →

Other small models

All small models →