Suchandra/bengali_language_NER

Bengali Named Entity Recognition

Fine-tuning bert-base-multilingual-cased on Wikiann dataset for performing NER on Bengali language.

Label ID and its corresponding label name

Label ID	Label Name
0	O
1	B-PER
2	I-PER
3	B-ORG
4	I-ORG
5	B-LOC
6	I-LOC

Results

Name	Overall F1	LOC F1	ORG F1	PER F1
Train set	0.997927	0.998246	0.996613	0.998769
Validation set	0.970187	0.969212	0.956831	0.982079
Test set	0.9673011	0.967120	0.963614	0.970938

Example

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("Suchandra/bengali_language_NER")
model = AutoModelForTokenClassification.from_pretrained("Suchandra/bengali_language_NER")

nlp = pipeline("ner", model=model, tokenizer=tokenizer)
example = "মারভিন দি মারসিয়ান"

ner_results = nlp(example)
ner_results