MFQA
MQA is a Multilingual corpus of Questions and Answers (MQA) parsed from the Common Crawl. Questions are divided in two types: Frequently Asked Questions (FAQ) and Community Question Answering (CQA).
University of Antwerp
MFAQ: a Multilingual FAQ Dataset
frequently asked questions, question answering
nan
Maxime De Bruyn, Ehsan Lotfi, Jeska Buhmann, Walter Daelemans
In this paper, we present the first multilingual
FAQ dataset publicly available. We collected
around 6M FAQ pairs from the web, in 21 different languages. Although this is significantly
larger than existing FAQ retrieval datasets, it
comes with its own challenges: duplication of
content and uneven distribution of topics. We
adopt a similar setup as Dense Passage Retrieval (DPR) (Karpukhin et al., 2020) and test
various bi-encoders on this dataset. Our experiments reveal that a multilingual model based
on XLM-RoBERTa (Conneau et al., 2019)
achieves the best results, except for English.
Lower resources languages seem to learn from
one another as a multilingual model achieves a
higher MRR than language-specific ones. Our
qualitative analysis reveals the brittleness of
the model on simple word changes. We publicly release our dataset1
, model2
and training
script