metadata
license: cc-by-4.0
language:
- en
datasets:
- CoNLL2003/AIDA
- Wikipedia
- sshavara/AIDA_testc
tags:
- SpEL
- Entity Linking
- Structured Prediction
widget:
- text: Leicestershire beat Somerset by an innings and 39 runs in two days.
SpEL (Structured prediction for Entity Linking)
SpEL model finetuned on English Wikipedia as well as the training portion of CoNLL2003/AIDA. It is introduced in the paper SPEL: Structured Prediction for Entity Linking (EMNLP 2023). The code and data are available in this repository.
Usage
The following snippet demonstrates a quick way that SpEL can be used to generate subword-level, word-level, and phrase-level annotations for a sentence.
# download SpEL from https://github.com/shavarani/SpEL
from transformers import AutoTokenizer
from spel.model import SpELAnnotator, dl_sa
from spel.configuration import device
from spel.utils import get_subword_to_word_mapping
from spel.span_annotation import WordAnnotation, PhraseAnnotation
finetuned_after_step = 4
sentence = "Grace Kelly by Mika reached the top of the UK Singles Chart in 2007."
tokenizer = AutoTokenizer.from_pretrained("roberta-base")
# ############################################# LOAD SpEL #############################################################
spel = SpELAnnotator()
spel.init_model_from_scratch(device=device)
if finetuned_after_step == 3:
spel.shrink_classification_head_to_aida(device)
spel.load_checkpoint(None, device=device, load_from_torch_hub=True, finetuned_after_step=finetuned_after_step)
# ############################################# RUN SpEL ##############################################################
inputs = tokenizer(sentence, return_tensors="pt")
token_offsets = list(zip(inputs.encodings[0].tokens,inputs.encodings[0].offsets))
subword_annotations = spel.annotate_subword_ids(inputs.input_ids, k_for_top_k_to_keep=10, token_offsets=token_offsets)
# #################################### CREATE WORD-LEVEL ANNOTATIONS ##################################################
tokens_offsets = token_offsets[1:-1]
subword_annotations = subword_annotations[1:]
for sa in subword_annotations:
sa.idx2tag = dl_sa.mentions_itos
word_annotations = [WordAnnotation(subword_annotations[m[0]:m[1]], tokens_offsets[m[0]:m[1]])
for m in get_subword_to_word_mapping(inputs.tokens(), sentence)]
# ################################## CREATE PHRASE-LEVEL ANNOTATIONS ##################################################
phrase_annotations = []
for w in word_annotations:
if not w.annotations:
continue
if phrase_annotations and phrase_annotations[-1].resolved_annotation == w.resolved_annotation:
phrase_annotations[-1].add(w)
else:
phrase_annotations.append(PhraseAnnotation(w))
# ################################## PRINT OUT THE CREATED ANNOTATIONS ################################################
for phrase_annotation in phrase_annotations:
print(dl_sa.mentions_itos[phrase_annotation.resolved_annotation])
Evaluation Results
Entity Linking evaluation results of SpEL compared to that of the literature over AIDA test sets:
| Approach | EL Micro-F1 test-a |
EL Micro-F1 test-b |
#params on GPU |
speed sec/doc |
|---|---|---|---|---|
| Hoffart et al. (2011) | 72.4 | 72.8 | - | - |
| Kolitsas et al. (2018) | 89.4 | 82.4 | 330.7M | 0.097 |
| Broscheit (2019) | 86.0 | 79.3 | 495.1M | 0.613 |
| Peters et al. (2019) | 82.1 | 73.1 | - | - |
| Martins et al. (2019) | 85.2 | 81.9 | - | - |
| van Hulst et al. (2020) | 83.3 | 82.4 | 19.0M | 0.337 |
| Févry et al. (2020) | 79.7 | 76.7 | - | - |
| Poerner et al. (2020) | 90.8 | 85.0 | 131.1M | - |
| Kannan Ravi et al. (2021) | - | 83.1 | - | - |
| De Cao et al. (2021b) | - | 83.7 | 406.3M | 40.969 |
| De Cao et al. (2021a) (no mention-specific candidate set) |
61.9 | 49.4 | 124.8M | 0.268 |
| De Cao et al. (2021a) (using PPRforNED candidate set) |
90.1 | 85.5 | 124.8M | 0.194 |
| Mrini et al. (2022) | - | 85.7 | (train) 811.5M (test) 406.2M |
- |
| Zhang et al. (2022) | - | 85.8 | 1004.3M | - |
| Feng et al. (2022) | - | 86.3 | 157.3M | - |
| SpEL-base (no mention-specific candidate set) | 91.3 | 85.5 | 128.9M | 0.084 |
| SpEL-base (KB+Yago candidate set) | 90.6 | 85.7 | 128.9M | 0.158 |
| SpEL-base (PPRforNED candidate set) (context-agnostic) |
91.7 | 86.8 | 128.9M | 0.153 |
| SpEL-base (PPRforNED candidate set) (context-aware) |
92.7 | 88.1 | 128.9M | 0.156 |
| SpEL-large (no mention-specific candidate set) | 91.6 | 85.8 | 361.1M | 0.273 |
| SpEL-large (KB+Yago candidate set) | 90.8 | 85.7 | 361.1M | 0.267 |
| SpEL-large (PPRforNED candidate set) (context-agnostic) |
92.0 | 87.3 | 361.1M | 0.268 |
| SpEL-large (PPRforNED candidate set) (context-aware) |
92.9 | 88.6 | 361.1M | 0.267 |
Citation
If you use SpEL finetuned models or data, please cite our paper:
@inproceedings{shavarani2023spel,
title={Sp{EL}: Structured Prediction for Entity Linking},
author={Shavarani, Hassan S. and Sarkar, Anoop},
booktitle={The 2023 Conference on Empirical Methods in Natural Language Processing},
year={2023},
url={https://arxiv.org/abs/2310.14684}
}