LEIA/LEIA-multilingual

This is an emotion classification model based on fine-tuning of a Bernice model, which is a pre-trained model trained on multilingual Twitter data. The fine-tuning dataset is a subset of the self-labeled emotion dataset (Lykousas et al., 2019) in English that corresponds to Anger, Fear, Sadness, Joy, and Affection. See the paper, LEIA: Linguistic Embeddings for the Identification of Affect for further details.

Evaluation

We evaluated LEIA-multilingual on posts with self-annotated emotion labels identified as non-English using an ensemble of language identification tools. The table below shows the macro-F1 scores aggregated across emotion categories for each language:

Language	Macro-F1
ar	44.18[43.07,45.29]
da	65.44[60.96,69.83]
de	60.47[57.58,63.38]
es	61.67[60.79,62.55]
fi	45.1[40.96,49.14]
fr	65.78[63.19,68.36]
it	63.37[59.67,67.1]
pt	57.27[55.15,59.4]
tl	58.37[55.51,61.23]
tr	45.42[41.17,49.79]

Citation

Please cite the following paper if you find the model useful for your work:

@article{aroyehun2023leia,
  title={LEIA: Linguistic Embeddings for the Identification of Affect},
  author={Aroyehun, Segun Taofeek and Malik, Lukas and Metzler, Hannah and Haimerl, Nikolas and Di Natale, Anna and Garcia, David},
  journal={EPJ Data Science},
  volume={12},
  year={2023},
  publisher={Springer}
}