Update README.md
Browse files
README.md
CHANGED
@@ -12,24 +12,28 @@ widget:
|
|
12 |
- text: "You wont believe what happened to me today :)"
|
13 |
- text: "You wont believe what happened to me today :("
|
14 |
---
|
15 |
-
This is an emotion classification model based on
|
16 |
See the paper, [LEIA: Linguistic Embeddings for the Identification of Affect](https://arxiv.org/abs/2304.10973) for further details.
|
17 |
|
18 |
## Evaluation
|
19 |
-
We evaluated LEIA-multilingual on Vent posts with self-annotated emotion labels
|
20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
21 |
|
22 |
-
language | Macro-F1
|
23 |
-
ar | 44.18[43.07,45.29]
|
24 |
-
da |65.44[60.96,69.83]
|
25 |
-
de |60.47[57.58,63.38]
|
26 |
-
es |61.67[60.79,62.55]
|
27 |
-
fi |45.1[40.96,49.14]
|
28 |
-
fr |65.78[63.19,68.36]
|
29 |
-
it |63.37[59.67,67.1]
|
30 |
-
pt |57.27[55.15,59.4]
|
31 |
-
tl |58.37[55.51,61.23]
|
32 |
-
tr |45.42[41.17,49.79]
|
33 |
|
34 |
|
35 |
## Citation
|
|
|
12 |
- text: "You wont believe what happened to me today :)"
|
13 |
- text: "You wont believe what happened to me today :("
|
14 |
---
|
15 |
+
This is an emotion classification model based on fine-tuning of a Bernice model (which is a multilingual pre-trained model trained on multilingual Twitter data) on self-labeled emotion dataset (Lykousas et al., 2019) in English that corresponds to Anger, Fear, Sadness, Joy, and Affection.
|
16 |
See the paper, [LEIA: Linguistic Embeddings for the Identification of Affect](https://arxiv.org/abs/2304.10973) for further details.
|
17 |
|
18 |
## Evaluation
|
19 |
+
We evaluated LEIA-multilingual on Vent posts with self-annotated emotion labels identified as non-English using an ensemble of language identefication tools.
|
20 |
+
The table below shows the macro-F1 scores across emotion categories and languages:
|
21 |
+
|
22 |
+
|
23 |
+
|
24 |
+
|Language|Macro-F1|
|
25 |
+
|:---:|:---:|
|
26 |
+
|ar |44.18[43.07,45.29]|
|
27 |
+
|da |65.44[60.96,69.83] |
|
28 |
+
|de |60.47[57.58,63.38] |
|
29 |
+
|es |61.67[60.79,62.55] |
|
30 |
+
|fi |45.1[40.96,49.14] |
|
31 |
+
|fr |65.78[63.19,68.36] |
|
32 |
+
|it |63.37[59.67,67.1] |
|
33 |
+
|pt |57.27[55.15,59.4] |
|
34 |
+
|tl |58.37[55.51,61.23] |
|
35 |
+
|tr |45.42[41.17,49.79]|
|
36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
|
38 |
|
39 |
## Citation
|