Spaces:
Running
on
CPU Upgrade
Adding German Faithfulness Detection Task
Hi,
We think it would beneficial to add more languages to capture multi lingual performance for hallucination detection tasks. We released a benchmark for faithfulness detection in German text summarization: https://github.com/mediatechnologycenter/Absinth. We will also soon release the corresponding paper, which has been accepted for Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING).
Would it be possible to add this task to the leader-board?
Please reach out to us, if you have any questions.
Thank you :)
Hey @mtc ! We decided to stay away from multi-lingual benchmarks at the moment ( @pingnieuk is also very fond of these) since I think we already have a ton of datasets and tasks, and compute is a precious resource nowadays :)