LEIA
/

saroyehun commited on
Commit
80be514
1 Parent(s): 39ea243

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -14
README.md CHANGED
@@ -12,24 +12,28 @@ widget:
12
  - text: "You wont believe what happened to me today :)"
13
  - text: "You wont believe what happened to me today :("
14
  ---
15
- This is an emotion classification model based on finetuning of a Bernice model (a multilingual pre-trained model trained on multilingual Twitter data) on self-labeled emotion dataset (Lykousas et al., 2019) in English that corresponds to Anger, Fear, Sadness, Joy, and Affection.
16
  See the paper, [LEIA: Linguistic Embeddings for the Identification of Affect](https://arxiv.org/abs/2304.10973) for further details.
17
 
18
  ## Evaluation
19
- We evaluated LEIA-multilingual on Vent posts with self-annotated emotion labels that was identified (using an ensemble of language identefication tools) to be non-English.
20
- See the below for the macro-F1 scores across emotion categories and languages:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
- language | Macro-F1
23
- ar | 44.18[43.07,45.29]
24
- da |65.44[60.96,69.83]
25
- de |60.47[57.58,63.38]
26
- es |61.67[60.79,62.55]
27
- fi |45.1[40.96,49.14]
28
- fr |65.78[63.19,68.36]
29
- it |63.37[59.67,67.1]
30
- pt |57.27[55.15,59.4]
31
- tl |58.37[55.51,61.23]
32
- tr |45.42[41.17,49.79]
33
 
34
 
35
  ## Citation
 
12
  - text: "You wont believe what happened to me today :)"
13
  - text: "You wont believe what happened to me today :("
14
  ---
15
+ This is an emotion classification model based on fine-tuning of a Bernice model (which is a multilingual pre-trained model trained on multilingual Twitter data) on self-labeled emotion dataset (Lykousas et al., 2019) in English that corresponds to Anger, Fear, Sadness, Joy, and Affection.
16
  See the paper, [LEIA: Linguistic Embeddings for the Identification of Affect](https://arxiv.org/abs/2304.10973) for further details.
17
 
18
  ## Evaluation
19
+ We evaluated LEIA-multilingual on Vent posts with self-annotated emotion labels identified as non-English using an ensemble of language identefication tools.
20
+ The table below shows the macro-F1 scores across emotion categories and languages:
21
+
22
+
23
+
24
+ |Language|Macro-F1|
25
+ |:---:|:---:|
26
+ |ar |44.18[43.07,45.29]|
27
+ |da |65.44[60.96,69.83] |
28
+ |de |60.47[57.58,63.38] |
29
+ |es |61.67[60.79,62.55] |
30
+ |fi |45.1[40.96,49.14] |
31
+ |fr |65.78[63.19,68.36] |
32
+ |it |63.37[59.67,67.1] |
33
+ |pt |57.27[55.15,59.4] |
34
+ |tl |58.37[55.51,61.23] |
35
+ |tr |45.42[41.17,49.79]|
36
 
 
 
 
 
 
 
 
 
 
 
 
37
 
38
 
39
  ## Citation