X-CSR
automatically translate the original CSQA and CODAH datasets, which only have English versions, to 15 other languages, forming development and test sets for studying X-CSR
University of Southern California
Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning
commonsense reasoning
nan
Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, Xiang Ren

Commonsense reasoning research has so far
been mainly limited to English. We aim
to evaluate and improve popular multilingual
language models (ML-LMs) to help advance
commonsense reasoning (CSR) beyond English. We collect the Mickey corpus, consisting of 561k sentences in 11 different languages, which can be used for analyzing and
improving ML-LMs. We propose Mickey
Probe, a language-agnostic probing task for
fairly evaluating the common sense of popular ML-LMs across different languages. Also,
we create two new datasets, X-CSQA and XCODAH, by translating their English versions
to 15 other languages, so that we can evaluate
popular ML-LMs for cross-lingual commonsense reasoning. To improve the performance
beyond English, we propose a simple yet effective method — multilingual contrastive pretraining (MCP). It significantly enhances sentence representations, yielding a large performance gain on both benchmarks (e.g., +2.7%
accuracy for X-CSQA over XLM-RL).