language:
- multilingual
- af
- am
- ar
- as
- az
- be
- bg
- bm
- bn
- br
- bs
- ca
- cs
- cy
- da
- de
- el
- en
- eo
- es
- et
- eu
- fa
- ff
- fi
- fr
- fy
- ga
- gd
- gl
- gn
- gu
- ha
- he
- hi
- hr
- ht
- hu
- hy
- id
- ig
- is
- it
- ja
- jv
- ka
- kg
- kk
- km
- kn
- ko
- ku
- ky
- la
- lg
- ln
- lo
- lt
- lv
- mg
- mk
- ml
- mn
- mr
- ms
- my
- ne
- nl
- 'no'
- om
- or
- pa
- pl
- ps
- pt
- qu
- ro
- ru
- sa
- sd
- si
- sk
- sl
- so
- sq
- sr
- ss
- su
- sv
- sw
- ta
- te
- th
- ti
- tl
- tn
- tr
- uk
- ur
- uz
- vi
- wo
- xh
- yo
- zh
tags:
- retrieval
- entity-retrieval
- named-entity-disambiguation
- entity-disambiguation
- named-entity-linking
- entity-linking
- text2text-generation
mGENRE
The mGENRE (multilingual Generative ENtity REtrieval) system as presented in Multilingual Autoregressive Entity Linking implemented in pytorch.
In a nutshell, mGENRE uses a sequence-to-sequence approach to entity retrieval (e.g., linking), based on fine-tuned mBART architecture. GENRE performs retrieval generating the unique entity name conditioned on the input text using constrained beam search to only generate valid identifiers. The model was first released in the facebookresearch/GENRE repository using fairseq (the transformers models are obtained with a conversion script similar to this.
This model was trained on 105 languages from Wikipedia.
BibTeX entry and citation info
Please consider citing our works if you use code from this repository.
@article{decao2020multilingual,
author = {De Cao, Nicola and Wu, Ledell and Popat, Kashyap and Artetxe, Mikel
and Goyal, Naman and Plekhanov, Mikhail and Zettlemoyer, Luke
and Cancedda, Nicola and Riedel, Sebastian and Petroni, Fabio},
title = "{Multilingual Autoregressive Entity Linking}",
journal = {Transactions of the Association for Computational Linguistics},
volume = {10},
pages = {274-290},
year = {2022},
month = {03},
issn = {2307-387X},
doi = {10.1162/tacl_a_00460},
url = {https://doi.org/10.1162/tacl\_a\_00460},
eprint = {https://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl\_a\_00460/2004070/tacl\_a\_00460.pdf},
}
Usage
Here is an example of generation for Wikipedia page disambiguation:
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("impresso-project/nel-historic-multilingual")
model = AutoModelForSeq2SeqLM.from_pretrained("impresso-project/nel-historic-multilingual").eval()
sentences = ["[START] United Press [END] - On the home front, the British populace remains steadfast in the face of ongoing air raids. In [START] London [END], despite the destruction, the spirit of the people is unbroken, with volunteers and civil defense units working tirelessly to support the war effort. Reports from [START] BUP [START]correspondents highlight the nationwide push for increased production in factories, essential for supplying the front lines with the materials needed for victory. "]
outputs = model.generate(
**tokenizer(sentences, return_tensors="pt"),
num_beams=5,
num_return_sequences=5
)
tokenizer.batch_decode(outputs, skip_special_tokens=True)
which outputs the following top-5 predictions (using constrained beam search)
['Albert Einstein >> it',
'Albert Einstein (disambiguation) >> en',
'Alfred Einstein >> it',
'Alberto Einstein >> it',
'Einstein >> it']