SpanMarker with xlm-roberta-base

Trained on various nordic lang. datasets: see https://huggingface.co/datasets/tollefj/nordic-ner

This is a SpanMarker model trained on the norne dataset that can be used for Named Entity Recognition. This SpanMarker model uses FacebookAI/xlm-roberta-base as the underlying encoder.

Model Details

Model Description

Model Type: SpanMarker
Encoder: FacebookAI/xlm-roberta-base
Maximum Sequence Length: 256 tokens
Maximum Entity Length: 8 words
Training Dataset: norne
Language: en
License: cc-by-sa-4.0

Model Sources

Repository: SpanMarker on GitHub
Thesis: SpanMarker For Named Entity Recognition

Model Labels

Label	Examples
LOC	"Gran", "Leicestershire", "Den tyske antarktisekspedisjonen"
MISC	"socialdemokratiske", "nationalist", "Living Legend"
ORG	"Stabæk", "Samlaget", "Marillion"
PER	"Fish", "Dmitrij Medvedev", "Guru Ardjan Dev"

Evaluation

Metrics

Label	Precision	Recall	F1
all	0.9218	0.9146	0.9182
LOC	0.9284	0.9433	0.9358
MISC	0.6515	0.6047	0.6272
ORG	0.8951	0.8547	0.8745
PER	0.9513	0.9526	0.9520

Uses

Direct Use for Inference

from span_marker import SpanMarkerModel

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("span_marker_model_id")
# Run inference
entities = model.predict("Roddarn blir proffs efter OS.")

Downstream Use

You can finetune this model on your own dataset.

Click to expand

from span_marker import SpanMarkerModel, Trainer

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("span_marker_model_id")

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("span_marker_model_id-finetuned")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Sentence length	1	12.8175	331
Entities per sentence	0	1.0055	54

Training Hyperparameters

learning_rate: 5e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3

Training Results

Epoch	Step	Validation Loss	Validation Precision	Validation Recall	Validation F1	Validation Accuracy
0.5711	3000	0.0146	0.8650	0.8725	0.8687	0.9722
1.1422	6000	0.0123	0.8994	0.8920	0.8957	0.9778
1.7133	9000	0.0101	0.9184	0.8984	0.9083	0.9805
2.2844	12000	0.0101	0.9198	0.9110	0.9154	0.9818
2.8555	15000	0.0089	0.9245	0.9150	0.9197	0.9830

Framework Versions

Python: 3.12.2
SpanMarker: 1.5.0
Transformers: 4.38.2
PyTorch: 2.2.1+cu121
Datasets: 2.18.0
Tokenizers: 0.15.2

Citation

BibTeX

@software{Aarsen_SpanMarker,
    author = {Aarsen, Tom},
    license = {Apache-2.0},
    title = {{SpanMarker for Named Entity Recognition}},
    url = {https://github.com/tomaarsen/SpanMarkerNER}
}

tollefj
/

nordic-ner