xquad
a benchmark dataset for evaluating cross-lingual question answering performance. The dataset consists of a subset of 240 paragraphs and 1190 question-answer pairs from the development set of SQuAD v1.1 (Rajpurkar et al., 2016) together with their professional translations into ten languages: Spanish, German, Greek, Russian, Turkish, Arabic, Vietnamese, Thai, Chinese, and Hindi. Consequently, the dataset is entirely parallel across 11 languages.
DeepMind
On the Cross-lingual Transferability of Monolingual Representations
question answering
Association of Computation Linguistics
Mikel Artetxe†, Sebastian Ruder, Dani Yogatama

State-of-the-art unsupervised multilingual
models (e.g., multilingual BERT) have been
shown to generalize in a zero-shot crosslingual setting. This generalization ability has
been attributed to the use of a shared subword
vocabulary and joint training across multiple
languages giving rise to deep multilingual
abstractions. We evaluate this hypothesis by
designing an alternative approach that transfers a monolingual model to new languages
at the lexical level. More concretely, we first
train a transformer-based masked language
model on one language, and transfer it to a
new language by learning a new embedding
matrix with the same masked language
modeling objective—freezing parameters
of all other layers. This approach does not
rely on a shared vocabulary or joint training.
However, we show that it is competitive with
multilingual BERT on standard cross-lingual
classification benchmarks and on a new
Cross-lingual Question Answering Dataset
(XQuAD). Our results contradict common
beliefs of the basis of the generalization ability
of multilingual models and suggest that deep
monolingual models learn some abstractions
that generalize across languages. We also
release XQuAD as a more comprehensive
cross-lingual benchmark, which comprises
240 paragraphs and 1190 question-answer
pairs from SQuAD v1.1 translated into ten
languages by professional translators.