Update README.md
Browse files
README.md
CHANGED
|
@@ -24,6 +24,18 @@ Each branch contains an individual bits per weight, with the main one containing
|
|
| 24 |
|
| 25 |
Original model: https://huggingface.co/Locutusque/OpenCerebrum-2.0-7B
|
| 26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
| Branch | Bits | lm_head bits | VRAM (4k) | VRAM (16k) | VRAM (32k) | Description |
|
| 28 |
| ----- | ---- | ------- | ------ | ------ | ------ | ------------ |
|
| 29 |
| [8_0](https://huggingface.co/bartowski/OpenCerebrum-2.0-7B-exl2/tree/8_0) | 8.0 | 8.0 | 8.4 GB | 9.8 GB | 11.8 GB | Maximum quality that ExLlamaV2 can produce, near unquantized performance. |
|
|
|
|
| 24 |
|
| 25 |
Original model: https://huggingface.co/Locutusque/OpenCerebrum-2.0-7B
|
| 26 |
|
| 27 |
+
## Prompt format
|
| 28 |
+
|
| 29 |
+
No chat template specified so ChatML is used. This may be incorrect, check original model card for details.
|
| 30 |
+
|
| 31 |
+
```
|
| 32 |
+
<|im_start|>system
|
| 33 |
+
{message}<|im_end|>
|
| 34 |
+
<|im_start|>user
|
| 35 |
+
{user message}<|im_end|>
|
| 36 |
+
<|im_start|>assistant
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
| Branch | Bits | lm_head bits | VRAM (4k) | VRAM (16k) | VRAM (32k) | Description |
|
| 40 |
| ----- | ---- | ------- | ------ | ------ | ------ | ------------ |
|
| 41 |
| [8_0](https://huggingface.co/bartowski/OpenCerebrum-2.0-7B-exl2/tree/8_0) | 8.0 | 8.0 | 8.4 GB | 9.8 GB | 11.8 GB | Maximum quality that ExLlamaV2 can produce, near unquantized performance. |
|