Command R GGUF models
173 GGUF models in the Command R family, quantized by 70 uploaders. Sizes run from 29M to 160B parameters. The longest trained context is 256K tokens.
About the Command R GGUF models
Command R is represented on ggufmodels.com by 173 GGUF models, quantized by 70 uploaders on Hugging Face. The most downloaded are C4Ai Command R7B 12 2024, C4Ai Command R Plus 08 2024 and C4Ai Command R Plus 08 2024. Parameter counts run from 29M to 160B, and the longest trained context window is 256K tokens.
A Q4_K_M file of a Command R model is between 0.1 to 96.6 GB, which is the quant most people start with. Most files come from mradermacher (46), Lucy-in-the-Sky (11) and bartowski (8). The quants you will meet most often are Q4_K_M (114), Q2_K (99), Q6_K (96) and Q5_K_M (89), where the number is how many models offer that quant.
Frequently asked questions
Which Command R GGUF should I download?
The most downloaded Command R GGUF here is C4Ai Command R7B 12 2024. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.
How much VRAM does a Command R model need?
Q4_K_M files of Command R models on this site range from 0.1 to 96.6 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.
Where do Command R GGUF files come from?
The most common quantizers here are mradermacher, Lucy-in-the-Sky and bartowski. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.