palomapiot
commited on
Commit
•
5023ee4
1
Parent(s):
9442694
Update README.md
Browse files
README.md
CHANGED
@@ -14,33 +14,16 @@ tags:
|
|
14 |
# Mistral Fine-Tuned on not Engaging with Hate Speech
|
15 |
|
16 |
## Model Description
|
17 |
-
This model is a fine-tuned version of `mistralai/Mistral-7B-Instruct-v0.
|
18 |
|
19 |
## Intended Uses & Limitations
|
20 |
This model is intended for research purposes in conversational applications to stop hate speech generation.
|
21 |
|
22 |
-
|
23 |
-
- **Developed by:** [More Information Needed]
|
24 |
-
- **Funded by [optional]:** [More Information Needed]
|
25 |
-
- **Shared by [optional]:** [More Information Needed]
|
26 |
-
- **Model type:** [More Information Needed]
|
27 |
-
- **Language(s) (NLP):** [More Information Needed]
|
28 |
-
- **License:** [More Information Needed]
|
29 |
-
- **Finetuned from model [optional]:** [More Information Needed]
|
30 |
-
|
31 |
-
### Model Sources [optional]
|
32 |
-
|
33 |
-
<!-- Provide the basic links for the model. -->
|
34 |
-
|
35 |
-
- **Repository:** [More Information Needed]
|
36 |
-
- **Paper [optional]:** [More Information Needed]
|
37 |
-
|
38 |
-
|
39 |
## Bias, Risks, and Limitations
|
40 |
|
41 |
-
|
42 |
-
|
43 |
-
|
44 |
|
45 |
### Recommendations
|
46 |
|
@@ -74,32 +57,27 @@ Use the code below to get started with the model.
|
|
74 |
|
75 |
## Environmental Impact
|
76 |
|
77 |
-
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
78 |
-
|
79 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
80 |
|
81 |
-
- **Hardware Type:**
|
82 |
-
- **Hours used:**
|
83 |
-
- **Cloud Provider:**
|
84 |
-
- **
|
85 |
-
- **Carbon Emitted
|
86 |
-
|
87 |
|
88 |
-
## Citation [optional]
|
89 |
|
90 |
-
|
91 |
|
92 |
-
|
93 |
-
|
94 |
-
[More Information Needed]
|
95 |
|
96 |
-
|
97 |
-
|
98 |
-
|
|
|
|
|
99 |
|
100 |
## Training procedure
|
101 |
|
102 |
-
|
103 |
The following `bitsandbytes` quantization config was used during training:
|
104 |
- quant_method: bitsandbytes
|
105 |
- _load_in_8bit: False
|
@@ -117,5 +95,7 @@ The following `bitsandbytes` quantization config was used during training:
|
|
117 |
|
118 |
### Framework versions
|
119 |
|
|
|
120 |
|
121 |
-
|
|
|
|
14 |
# Mistral Fine-Tuned on not Engaging with Hate Speech
|
15 |
|
16 |
## Model Description
|
17 |
+
This model is a fine-tuned version of `mistralai/Mistral-7B-Instruct-v0.1` on a hate speech dataset using the PEFT approach, to prevent the model from exacerbating hate discourse.
|
18 |
|
19 |
## Intended Uses & Limitations
|
20 |
This model is intended for research purposes in conversational applications to stop hate speech generation.
|
21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
## Bias, Risks, and Limitations
|
23 |
|
24 |
+
- **Biases**: The model may carry biases present in the training data.
|
25 |
+
- **False Positives/Negatives**: It's not perfect and may continue some hate speech conversations.
|
26 |
+
- **Domain Specificity**: Performance may vary across different domains.
|
27 |
|
28 |
### Recommendations
|
29 |
|
|
|
57 |
|
58 |
## Environmental Impact
|
59 |
|
|
|
|
|
60 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
61 |
|
62 |
+
- **Hardware Type:** RTX A6000
|
63 |
+
- **Hours used:** 9
|
64 |
+
- **Cloud Provider:** Private Infrastructure
|
65 |
+
- **Carbon Efficiency (kg/kWh):** 0,432
|
66 |
+
- **Carbon Emitted (kg eq. CO2):** 1,17
|
|
|
67 |
|
|
|
68 |
|
69 |
+
## Citation
|
70 |
|
71 |
+
If you use this model, please cite the following reference:
|
|
|
|
|
72 |
|
73 |
+
```bibtex
|
74 |
+
@article{
|
75 |
+
SOON!
|
76 |
+
}
|
77 |
+
```
|
78 |
|
79 |
## Training procedure
|
80 |
|
|
|
81 |
The following `bitsandbytes` quantization config was used during training:
|
82 |
- quant_method: bitsandbytes
|
83 |
- _load_in_8bit: False
|
|
|
95 |
|
96 |
### Framework versions
|
97 |
|
98 |
+
- PEFT 0.6.2
|
99 |
|
100 |
+
## Acknowledgements
|
101 |
+
The authors thank the funding from the Horizon Europe research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 101073351. The authors also thank the financial support supplied by the Consellería de Cultura, Educación, Formación Profesional e Universidades (accreditation 2019-2022 ED431G/01, ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center in ICT of the University of A Coruña as a Research Center of the Galician University System and the project PID2022-137061OB-C21 (Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Proyectos de Generación de Conocimiento; supported by the European Regional Development Fund). The authors also thank the funding of project PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next Generation EU).
|