Update README.md
Browse files
README.md
CHANGED
@@ -5,19 +5,26 @@ tags:
|
|
5 |
- uncensored
|
6 |
|
7 |
---
|
8 |
-
openaccess-ai-collective Manticore-13b-Chat-Pyg with the Guanaco 13b qLoRa from TimDettmers applied, uncensored
|
9 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
---
|
11 |
|
12 |
-
|
|
|
13 |
|
14 |
Command used to quantize:
|
15 |
python3 llama.py Manticore-13B-Chat-Pyg-Guanaco-GPTQ-4bit-128g.no-act-order.safetensors c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors
|
16 |
|
17 |
|
18 |
|
19 |
-
The model seems to have benefited from further augmentation
|
20 |
Its capabilities seem broad, even compared with other Wizard or Manticore models, with expected weaknesses at coding. It is very good at in-context-learning and (in its class) reasoning.
|
|
|
21 |
Refreshingly, it does not seem to insist on aggressively sticking to narratives to justify formerly hallucinated output as much as similar models. It's output seems... eerily smart at times.
|
22 |
I believe the model is fully unrestricted/uncensored and will generally not berate.
|
23 |
|
@@ -26,6 +33,7 @@ I believe the model is fully unrestricted/uncensored and will generally not bera
|
|
26 |
Prompting style + settings:
|
27 |
---
|
28 |
Presumably due to the very diverse training-data the model accepts a variety of prompting styles with relatively few issues, including the ###-Variant, but seems to work best using:
|
|
|
29 |
|
30 |
user: "USER:" -
|
31 |
bot: "ASSISTANT:" -
|
|
|
5 |
- uncensored
|
6 |
|
7 |
---
|
|
|
8 |
---
|
9 |
+
# 4bit GPTQ of:
|
10 |
+
Manticore-13b-Chat-Pyg by [openaccess-ai-collective](https://huggingface.co/openaccess-ai-collective/manticore-13b-chat-pyg) with the Guanaco 13b qLoRa by [TimDettmers](https://huggingface.co/timdettmers/guanaco-13b) applied through [Monero](https://huggingface.co/Monero/Manticore-13b-Chat-Pyg-Guanaco), quantized by [mindrage](https://huggingface.co/mindrage), uncensored
|
11 |
+
|
12 |
+
|
13 |
+
[link to GGML Version](https://huggingface.co/mindrage/Manticore-13B-Chat-Pyg-Guanaco-GGML-q4_0)
|
14 |
+
|
15 |
---
|
16 |
|
17 |
+
|
18 |
+
Quantized to 4bit GPTQ, groupsize 128, no-act-order.
|
19 |
|
20 |
Command used to quantize:
|
21 |
python3 llama.py Manticore-13B-Chat-Pyg-Guanaco-GPTQ-4bit-128g.no-act-order.safetensors c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors
|
22 |
|
23 |
|
24 |
|
25 |
+
The model seems to have noticeably benefited from further augmentation with the Guanaco qLora.
|
26 |
Its capabilities seem broad, even compared with other Wizard or Manticore models, with expected weaknesses at coding. It is very good at in-context-learning and (in its class) reasoning.
|
27 |
+
It both follows instructions well, and can be used as a chatbot.
|
28 |
Refreshingly, it does not seem to insist on aggressively sticking to narratives to justify formerly hallucinated output as much as similar models. It's output seems... eerily smart at times.
|
29 |
I believe the model is fully unrestricted/uncensored and will generally not berate.
|
30 |
|
|
|
33 |
Prompting style + settings:
|
34 |
---
|
35 |
Presumably due to the very diverse training-data the model accepts a variety of prompting styles with relatively few issues, including the ###-Variant, but seems to work best using:
|
36 |
+
# "Naming" the model works great by simply modifying the context. Substantial changes in its behaviour can be caused very simply by appending to "ASSISTANT:", like "ASSISTANT: After careful consideration, thinking step-by-step, my response is:"
|
37 |
|
38 |
user: "USER:" -
|
39 |
bot: "ASSISTANT:" -
|