Commit
•
b7e6fa6
1
Parent(s):
a628832
Add model sizes (#5)
Browse files- Add model sizes (1312b4aba1a7d4cc42b47b7b19aaa1d567826b07)
Co-authored-by: nanoflooder <nanoflooder@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -16,6 +16,24 @@ The PR has been approved, we should expect it to be merged shortly into the main
|
|
| 16 |
* How do I use imatrix quants? Just like any other GGUF, the `.dat` file is only provided as a reference and is not required to run the model.
|
| 17 |
* If your last resort is to use an IQ1 quant then go for IQ1_M.
|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
> C4AI Command R+ is an open weights research release of a 104B billion parameter model with highly advanced capabilities, this includes Retrieval Augmented Generation (RAG) and tool use to automate sophisticated tasks. The tool use in this model generation enables multi-step tool use which allows the model to combine multiple tools over multiple steps to accomplish difficult tasks. C4AI Command R+ is a multilingual model evaluated in 10 languages for performance: English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Arabic, and Simplified Chinese. Command R+ is optimized for a variety of use cases including reasoning, summarization, and question answering.
|
| 20 |
|
| 21 |
| Layers | Context | [Template](https://huggingface.co/CohereForAI/c4ai-command-r-plus#tool-use--multihop-capabilities) |
|
|
|
|
| 16 |
* How do I use imatrix quants? Just like any other GGUF, the `.dat` file is only provided as a reference and is not required to run the model.
|
| 17 |
* If your last resort is to use an IQ1 quant then go for IQ1_M.
|
| 18 |
|
| 19 |
+
| Quant | Size |
|
| 20 |
+
|--------|--------|
|
| 21 |
+
|iq1\_s | 23.2 |
|
| 22 |
+
|iq1\_m | 25.2 |
|
| 23 |
+
|iq2\_xxs| 28.6 |
|
| 24 |
+
|iq2\_xs | 31.6 |
|
| 25 |
+
|iq2\_s | 33.3 |
|
| 26 |
+
|iq2\_m | 36.0 |
|
| 27 |
+
|iq3\_xxs| 40.7 |
|
| 28 |
+
|iq3\_xs | 43.6 |
|
| 29 |
+
|iq3\_s | 46.0 |
|
| 30 |
+
|iq3\_m | 47.7 |
|
| 31 |
+
|iq4\_xs | 56.3 |
|
| 32 |
+
|q5\_k\_s| 71.8 |
|
| 33 |
+
|q6\_k | 85.1 |
|
| 34 |
+
|q8\_0 | 110.3 |
|
| 35 |
+
|fp16 | 207.8 |
|
| 36 |
+
|
| 37 |
> C4AI Command R+ is an open weights research release of a 104B billion parameter model with highly advanced capabilities, this includes Retrieval Augmented Generation (RAG) and tool use to automate sophisticated tasks. The tool use in this model generation enables multi-step tool use which allows the model to combine multiple tools over multiple steps to accomplish difficult tasks. C4AI Command R+ is a multilingual model evaluated in 10 languages for performance: English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Arabic, and Simplified Chinese. Command R+ is optimized for a variety of use cases including reasoning, summarization, and question answering.
|
| 38 |
|
| 39 |
| Layers | Context | [Template](https://huggingface.co/CohereForAI/c4ai-command-r-plus#tool-use--multihop-capabilities) |
|