Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
language:
|
3 |
+
- ru
|
4 |
+
|
5 |
+
tags:
|
6 |
+
- toxic comments classification
|
7 |
+
|
8 |
+
licenses:
|
9 |
+
- cc-by-nc-sa
|
10 |
+
---
|
11 |
+
|
12 |
+
## General concept of the model
|
13 |
+
|
14 |
+
This model is trained on the dataset of inappropriate messages of the Russian language. The concept of inappropriateness is described [in this article ](https://arxiv.org/abs/2103.05345) presented at the workshop for Balto-Slavic NLP at the EACL-2021 conference. Please note that this article describes the first version of the dataset, while the model is trained on the extended version of the dataset open-sourced on our [GitHub](https://github.com/skoltech-nlp/inappropriate-sensitive-topics/blob/main/Version2/appropriateness/Appropriateness.csv). The properties of the dataset is the same as the one described in the article, the only difference is the size.
|
15 |
+
|
16 |
+
The model was trained, validated and tested only on the samples with 100% confidence, which allowed to get the following metrics on test set:
|
17 |
+
|
18 |
+
| | precision | recall | f1-score | support |
|
19 |
+
|--------------|----------|--------|----------|---------|
|
20 |
+
| 0 | 0.92 | 0.93 | 0.93 | 7839 |
|
21 |
+
| 1 | 0.80 | 0.76 | 0.78 | 2726 |
|
22 |
+
| accuracy | | | 0.89 | 10565 |
|
23 |
+
| macro avg | 0.86 | 0.85 | 0.85 | 10565 |
|
24 |
+
| weighted avg | 0.89 | 0.89 | 0.89 | 10565 |
|
25 |
+
|
26 |
+
## Licensing Information
|
27 |
+
|
28 |
+
[Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License][cc-by-nc-sa].
|
29 |
+
|
30 |
+
[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]
|
31 |
+
|
32 |
+
[cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/
|
33 |
+
[cc-by-nc-sa-image]: https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png
|
34 |
+
|
35 |
+
## Citation
|
36 |
+
|
37 |
+
If you find this repository helpful, feel free to cite our publication:
|
38 |
+
|
39 |
+
```
|
40 |
+
@inproceedings{babakov-etal-2021-bsnlp,
|
41 |
+
title = "Detecting Inappropriate Messages on Sensitive Topics that Could Harm a Company's Reputation",
|
42 |
+
author = "Babakov, Nikolay and Logacheva, Varvara and Kozlova, Olga and Semenov, Nikita and Panchenko, Alexander",
|
43 |
+
booktitle = "To appear in the Proceedings of the 8th Workshop on Balto-Slavic Natural Language Processing",
|
44 |
+
month = April,
|
45 |
+
year = "2021",
|
46 |
+
address = "Kyiv, Ukraine"
|
47 |
+
}
|
48 |
+
```
|