Update README.md

7c52089 verified 3 months ago

34.9 kB

	---
	base_model: roberta-large
	datasets:
	- YurtsAI/named_entity_recognition_document_context
	language:
	- en
	library_name: span-marker
	metrics:
	- precision
	- recall
	- f1
	pipeline_tag: token-classification
	tags:
	- span-marker
	- token-classification
	- ner
	- named-entity-recognition
	- generated_from_span_marker_trainer
	widget:
	- text: '* * phone call transcript: university research paper discussion * * * * date:
	* * 09041942 * * time: * * 3:45 pm * * participants: * * dr. emily carter (ec)
	- principal investigator dr. john smith (js) - co-investigator--- * * ec: * *
	hey john, got a minute to discuss the latest draft of our paper on crispr-cas9?'
	- text: monday is a chill day – beach time at barceloneta and maybe some shopping
	at la rambla.
	- text: don't forget to fast for at least 8 hours before the procedure – that means
	no food or drink after midnight!
	- text: whether it's buying a house in 5 years, saving for a killer vacation next
	summer, or just building an emergency fund, write it down.
	- text: '- * * full integration: * * all recipes from the rbso must be incorporated
	into event menus by november 1, 2023.'
	model-index:
	- name: SpanMarker with roberta-large on YurtsAI/named_entity_recognition_document_context
	results:
	- task:
	type: token-classification
	name: Named Entity Recognition
	dataset:
	name: Unknown
	type: YurtsAI/named_entity_recognition_document_context
	split: eval
	metrics:
	- type: f1
	value: 0.8349078585045542
	name: F1
	- type: precision
	value: 0.8308950630296387
	name: Precision
	- type: recall
	value: 0.8389596015495296
	name: Recall
	---

	# SpanMarker with roberta-large on YurtsAI/named_entity_recognition_document_context

	This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model trained on the [YurtsAI/named_entity_recognition_document_context](https://huggingface.co/datasets/YurtsAI/named_entity_recognition_document_context) dataset that can be used for Named Entity Recognition. This SpanMarker model uses [roberta-large](https://huggingface.co/roberta-large) as the underlying encoder.

	## Model Details

	### Model Description
	- Model Type: SpanMarker
	- Encoder: [roberta-large](https://huggingface.co/roberta-large)
	- Maximum Sequence Length: 256 tokens
	- Maximum Entity Length: 11 words
	- Training Dataset: [YurtsAI/named_entity_recognition_document_context](https://huggingface.co/datasets/YurtsAI/named_entity_recognition_document_context)
	- Language: en
	<!-- - License: Unknown -->

	### Model Sources

	- Repository: [SpanMarker on GitHub](https://github.com/tomaarsen/SpanMarkerNER)
	- Thesis: [SpanMarker For Named Entity Recognition](https://raw.githubusercontent.com/tomaarsen/SpanMarkerNER/main/thesis.pdf)

	### Model Labels
	\| Label \| Examples \|
	\|:--------------------------------------------------\|:------------------------------------------------------------------------------------------------------------------------------\|
	\| DATETIME__absolute \| "14:00 hrs", "15th november 2023 at 10:00 am", "october 15th , 2023" \|
	\| DATETIME__authored \| "25 february 26", "sunday , 21 august , 1938", "1961-05-08" \|
	\| DATETIME__range \| "29th of oct. , 2023", "september 2021 to august 2023", "jan 2022 - dec 2022" \|
	\| DATETIME__relative \| "eod friday", "dec 15 , 11:59 pm", "10/15" \|
	\| GENERAL__art-broadcastprogram \| "stranger things", "live q & a", "product design concept sketchbook for kids" \|
	\| GENERAL__art-film \| "the crown", "kill bill", "stranger things" \|
	\| GENERAL__art-music \| \|
	\| GENERAL__art-other \| "statue of liberty", "broadway show", "wicked" \|
	\| GENERAL__art-painting \| "draw your dream house", "design a superhero costume" \|
	\| GENERAL__art-writtenart \| "optimization of quantum algorithms for cryptographic applications", "introduction to algorithms", "intro to cs '' by j. doe" \|
	\| GENERAL__building-airport \| "ory", "charles de gaulle", "cdg" \|
	\| GENERAL__building-hospital \| "green valley clinic", "department of oncology", "st. mary 's hospital" \|
	\| GENERAL__building-hotel \| "le jules verne", "hôtel ritz", "the beverly hills hotel" \|
	\| GENERAL__building-library \| "ancient library", "the grand library", "jefferson library" \|
	\| GENERAL__building-other \| "louvre museum", "engineering building", "eiffel tower" \|
	\| GENERAL__building-restaurant \| "l'ambroisie", "bella 's bistro", "in-n-out burger" \|
	\| GENERAL__building-sportsfacility \| "fenway" \|
	\| GENERAL__building-theater \| "gershwin theatre", "opera house", "broadway" \|
	\| GENERAL__event-attack/battle/war/militaryconflict \| "1863 battle of ridgefield", "battle of gettysburg", "war of 1812" \|
	\| GENERAL__event-other \| "annual science fair", "summer splash '23", "research methodology workshop" \|
	\| GENERAL__event-sportsevent \| "international olympiad in informatics", "ftx", "ioi" \|
	\| GENERAL__location-GPE \| "fr", "paris ,", "italy" \|
	\| GENERAL__location-bodiesofwater \| "river x", "river blue", "seine river" \|
	\| GENERAL__location-island \| "maldives", "similan islands", "ellis island" \|
	\| GENERAL__location-mountain \| "andes mountains", "swiss alps", "pine ridge" \|
	\| GENERAL__location-other \| "times square", "old market", "venice beach" \|
	\| GENERAL__location-park \| "central park", "ueno park", "universal studios" \|
	\| GENERAL__location-road/railway/highway/transit \| "i-95", "underground railroad", "hollywood walk of fame" \|
	\| GENERAL__organization-company \| "green earth organics", "xyz corporation", "north atlantic fisheries" \|
	\| GENERAL__organization-education \| "graduate school", "xyz", "xyz university" \|
	\| GENERAL__organization-government/governmentagency \| "department of economic development", "moe", "ministry of environment" \|
	\| GENERAL__organization-media/newspaper \| "pinterest", "yelp", "insta" \|
	\| GENERAL__organization-other \| "historical society", "grants office", "admissions committee" \|
	\| GENERAL__organization-religion \| "buddhist", "zen buddhist", "shinto" \|
	\| GENERAL__organization-showorganization \| "phare", "the soundbytes" \|
	\| GENERAL__organization-sportsteam \| "varsity soccer team", "red sox" \|
	\| GENERAL__other-astronomything \| \|
	\| GENERAL__other-award \| "team excellence award", "innovation award", "employee of the month" \|
	\| GENERAL__other-biologything \| "fodmap", "troponin i", "cmp" \|
	\| GENERAL__other-chemicalthing \| "co2", "pm2.5", "nitrate" \|
	\| GENERAL__other-currency \| "usd", "inr", "$ $ $" \|
	\| GENERAL__other-disease \| "mi", "irritable bowel syndrome", "myocardial infarction" \|
	\| GENERAL__other-educationaldegree \| "executive mba", "phd in quantum computing ,", "phd" \|
	\| GENERAL__other-god \| "inari", "athena", "inari taisha" \|
	\| GENERAL__other-language \| "french", "english", "spanish" \|
	\| GENERAL__other-law \| "cas", "clean air standards", "environmental protection act ( epa ) 2023" \|
	\| GENERAL__other-livingthing \| "eastern box turtle", "monarch butterfly", "western burrowing owl" \|
	\| GENERAL__other-medical \| "asa", "dapt", "clopidogrel" \|
	\| GENERAL__person-artist/author \| "carol", "picasso", "warhol" \|
	\| GENERAL__person-other \| "jamie", "sarah", "mark" \|
	\| GENERAL__person-politician \| "jane doe", "vespasian", "constantine i" \|
	\| GENERAL__person-scholar \| "dr. smith", "dr. lee", "dr. johnson" \|
	\| GENERAL__person-soldier \| "davis", "lt. sarah johnson", "col. r. johnson" \|
	\| GENERAL__product-airplane \| "hmmwvs", "uh-60s", "m1a2s" \|
	\| GENERAL__product-car \| "hmmwvs", "high mobility multipurpose wheeled vehicles", "mine-resistant ambush protected" \|
	\| GENERAL__product-food \| "pumpkin spice", "quinoa salad", "golden jubilee feast" \|
	\| GENERAL__product-game \| "stardew valley", "valorant", "call of duty : warzone" \|
	\| GENERAL__product-other \| "engagement metrics", "xj-200", "smart goal templates" \|
	\| GENERAL__product-ship \| "liberty island ferry", "hms victory", "thames river cruise" \|
	\| GENERAL__product-software \| "instagram", "svm", "r" \|
	\| GENERAL__product-train \| "n'ex", "shinkansen", "tgv" \|
	\| GENERAL__product-weapon \| "m1 abrams", "m4 carbine", "m4 carbines" \|

	## Evaluation

	### Metrics
	\| Label \| Precision \| Recall \| F1 \|
	\|:--------------------------------------------------\|:----------\|:-------\|:-------\|
	\| all \| 0.8309 \| 0.8390 \| 0.8349 \|
	\| DATETIME__absolute \| 0.8744 \| 0.8577 \| 0.8660 \|
	\| DATETIME__authored \| 0.9956 \| 0.9935 \| 0.9946 \|
	\| DATETIME__range \| 0.8451 \| 0.9262 \| 0.8838 \|
	\| DATETIME__relative \| 0.8266 \| 0.7498 \| 0.7863 \|
	\| GENERAL__art-broadcastprogram \| 0.6538 \| 0.6296 \| 0.6415 \|
	\| GENERAL__art-film \| 0.8 \| 1.0 \| 0.8889 \|
	\| GENERAL__art-music \| 0.0 \| 0.0 \| 0.0 \|
	\| GENERAL__art-other \| 0.625 \| 0.7143 \| 0.6667 \|
	\| GENERAL__art-painting \| 0.0 \| 0.0 \| 0.0 \|
	\| GENERAL__art-writtenart \| 0.7373 \| 0.8047 \| 0.7695 \|
	\| GENERAL__building-airport \| 0.8668 \| 0.9689 \| 0.9150 \|
	\| GENERAL__building-hospital \| 0.8378 \| 0.9323 \| 0.8826 \|
	\| GENERAL__building-hotel \| 0.7577 \| 0.8603 \| 0.8057 \|
	\| GENERAL__building-library \| 0.0 \| 0.0 \| 0.0 \|
	\| GENERAL__building-other \| 0.7597 \| 0.8409 \| 0.7982 \|
	\| GENERAL__building-restaurant \| 0.7953 \| 0.8695 \| 0.8307 \|
	\| GENERAL__building-sportsfacility \| 0.0 \| 0.0 \| 0.0 \|
	\| GENERAL__building-theater \| 0.6 \| 0.6667 \| 0.6316 \|
	\| GENERAL__event-attack/battle/war/militaryconflict \| 0.8438 \| 0.9310 \| 0.8852 \|
	\| GENERAL__event-other \| 0.6019 \| 0.6382 \| 0.6195 \|
	\| GENERAL__event-sportsevent \| 0.0 \| 0.0 \| 0.0 \|
	\| GENERAL__location-GPE \| 0.7232 \| 0.7888 \| 0.7546 \|
	\| GENERAL__location-bodiesofwater \| 0.6724 \| 0.975 \| 0.7959 \|
	\| GENERAL__location-island \| 0.7455 \| 0.9111 \| 0.8200 \|
	\| GENERAL__location-mountain \| 0.7436 \| 0.8529 \| 0.7945 \|
	\| GENERAL__location-other \| 0.7186 \| 0.7793 \| 0.7477 \|
	\| GENERAL__location-park \| 0.7899 \| 0.8704 \| 0.8282 \|
	\| GENERAL__location-road/railway/highway/transit \| 0.6325 \| 0.7095 \| 0.6688 \|
	\| GENERAL__organization-company \| 0.8665 \| 0.8605 \| 0.8635 \|
	\| GENERAL__organization-education \| 0.8256 \| 0.8608 \| 0.8428 \|
	\| GENERAL__organization-government/governmentagency \| 0.8344 \| 0.8318 \| 0.8331 \|
	\| GENERAL__organization-media/newspaper \| 0.6667 \| 0.4 \| 0.5 \|
	\| GENERAL__organization-other \| 0.7790 \| 0.8105 \| 0.7944 \|
	\| GENERAL__organization-religion \| 0.6667 \| 0.8 \| 0.7273 \|
	\| GENERAL__organization-showorganization \| 0.0 \| 0.0 \| 0.0 \|
	\| GENERAL__organization-sportsteam \| 0.0 \| 0.0 \| 0.0 \|
	\| GENERAL__other-astronomything \| 0.0 \| 0.0 \| 0.0 \|
	\| GENERAL__other-award \| 0.8216 \| 0.8859 \| 0.8525 \|
	\| GENERAL__other-biologything \| 0.7246 \| 0.8961 \| 0.8013 \|
	\| GENERAL__other-chemicalthing \| 0.7687 \| 0.8047 \| 0.7863 \|
	\| GENERAL__other-currency \| 0.6304 \| 0.6744 \| 0.6517 \|
	\| GENERAL__other-disease \| 0.8594 \| 0.9048 \| 0.8815 \|
	\| GENERAL__other-educationaldegree \| 0.7119 \| 0.75 \| 0.7304 \|
	\| GENERAL__other-god \| 0.8 \| 0.5714 \| 0.6667 \|
	\| GENERAL__other-language \| 0.6818 \| 1.0 \| 0.8108 \|
	\| GENERAL__other-law \| 0.7978 \| 0.8462 \| 0.8212 \|
	\| GENERAL__other-livingthing \| 0.7385 \| 0.9320 \| 0.8240 \|
	\| GENERAL__other-medical \| 0.7778 \| 0.8343 \| 0.8050 \|
	\| GENERAL__person-artist/author \| 0.625 \| 0.3846 \| 0.4762 \|
	\| GENERAL__person-other \| 0.8839 \| 0.8979 \| 0.8908 \|
	\| GENERAL__person-politician \| 0.7534 \| 0.7432 \| 0.7483 \|
	\| GENERAL__person-scholar \| 0.8640 \| 0.8769 \| 0.8704 \|
	\| GENERAL__person-soldier \| 0.7674 \| 0.7586 \| 0.7630 \|
	\| GENERAL__product-airplane \| 0.6774 \| 0.6364 \| 0.6562 \|
	\| GENERAL__product-car \| 0.9286 \| 0.7879 \| 0.8525 \|
	\| GENERAL__product-food \| 0.7798 \| 0.7859 \| 0.7828 \|
	\| GENERAL__product-game \| 0.75 \| 0.75 \| 0.75 \|
	\| GENERAL__product-other \| 0.7175 \| 0.7537 \| 0.7351 \|
	\| GENERAL__product-ship \| 0.0 \| 0.0 \| 0.0 \|
	\| GENERAL__product-software \| 0.8093 \| 0.8403 \| 0.8245 \|
	\| GENERAL__product-train \| 0.75 \| 0.375 \| 0.5 \|
	\| GENERAL__product-weapon \| 0.7794 \| 0.8833 \| 0.8281 \|

	## Uses

	### Direct Use for Inference

	```python
	from span_marker import SpanMarkerModel

	# Download from the 🤗 Hub
	model = SpanMarkerModel.from_pretrained("YurtsAI/named_entity_recognition_document_context")
	# Run inference
	entities = model.predict("monday is a chill day – beach time at barceloneta and maybe some shopping at la rambla.")
	```

	### Downstream Use
	You can finetune this model on your own dataset.

	<details><summary>Click to expand</summary>

	```python
	from span_marker import SpanMarkerModel, Trainer

	# Download from the 🤗 Hub
	model = SpanMarkerModel.from_pretrained("YurtsAI/ner-document-context")

	# Specify a Dataset with "tokens" and "ner_tag" columns
	dataset = load_dataset("conll2003") # For example CoNLL2003

	# Initialize a Trainer using the pretrained model & dataset
	trainer = Trainer(
	model=model,
	train_dataset=dataset["train"],
	eval_dataset=dataset["validation"],
	)
	trainer.train()
	trainer.save_model("YurtsAI/named_entity_recognition_document_context-finetuned")
	```
	</details>

	<!--
	### Out-of-Scope Use

	List how the model may foreseeably be misused and address what users ought not to do with the model.
	-->

	<!--
	## Bias, Risks and Limitations

	What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.
	-->

	<!--
	### Recommendations

	What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.
	-->

	## Training Details

	### Training Set Metrics
	\| Training set \| Min \| Median \| Max \|
	\|:----------------------\|:----\|:--------\|:----\|
	\| Sentence length \| 1 \| 14.6796 \| 691 \|
	\| Entities per sentence \| 0 \| 0.4235 \| 35 \|

	### Training Hyperparameters
	- learning_rate: 1e-05
	- train_batch_size: 32
	- eval_batch_size: 32
	- seed: 42
	- gradient_accumulation_steps: 2
	- total_train_batch_size: 64
	- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
	- lr_scheduler_type: linear
	- lr_scheduler_warmup_ratio: 0.1
	- num_epochs: 3

	### Training Results
	\| Epoch \| Step \| Validation Loss \| Validation Precision \| Validation Recall \| Validation F1 \| Validation Accuracy \|
	\|:------:\|:-----:\|:---------------:\|:--------------------:\|:-----------------:\|:-------------:\|:-------------------:\|
	\| 0.0299 \| 500 \| 0.0254 \| 0.5244 \| 0.0116 \| 0.0228 \| 0.9292 \|
	\| 0.0597 \| 1000 \| 0.0144 \| 0.5380 \| 0.3492 \| 0.4235 \| 0.9444 \|
	\| 0.0896 \| 1500 \| 0.0099 \| 0.7134 \| 0.4410 \| 0.5450 \| 0.9534 \|
	\| 0.1194 \| 2000 \| 0.0088 \| 0.6461 \| 0.6571 \| 0.6516 \| 0.9596 \|
	\| 0.1493 \| 2500 \| 0.0074 \| 0.7177 \| 0.6363 \| 0.6745 \| 0.9628 \|
	\| 0.1791 \| 3000 \| 0.0075 \| 0.6612 \| 0.7342 \| 0.6958 \| 0.9637 \|
	\| 0.2090 \| 3500 \| 0.0073 \| 0.6686 \| 0.7286 \| 0.6973 \| 0.9634 \|
	\| 0.2388 \| 4000 \| 0.0061 \| 0.7552 \| 0.7044 \| 0.7289 \| 0.9693 \|
	\| 0.2687 \| 4500 \| 0.0062 \| 0.7385 \| 0.7150 \| 0.7266 \| 0.9682 \|
	\| 0.2986 \| 5000 \| 0.0070 \| 0.6667 \| 0.7792 \| 0.7186 \| 0.9654 \|
	\| 0.3284 \| 5500 \| 0.0063 \| 0.6984 \| 0.7774 \| 0.7358 \| 0.9689 \|
	\| 0.3583 \| 6000 \| 0.0055 \| 0.7941 \| 0.7023 \| 0.7454 \| 0.9706 \|
	\| 0.3881 \| 6500 \| 0.0055 \| 0.7540 \| 0.7640 \| 0.7589 \| 0.9722 \|
	\| 0.4180 \| 7000 \| 0.0053 \| 0.7700 \| 0.7614 \| 0.7657 \| 0.9732 \|
	\| 0.4478 \| 7500 \| 0.0053 \| 0.7791 \| 0.7698 \| 0.7744 \| 0.9742 \|
	\| 0.4777 \| 8000 \| 0.0054 \| 0.7396 \| 0.8062 \| 0.7715 \| 0.9729 \|
	\| 0.5075 \| 8500 \| 0.0051 \| 0.7653 \| 0.7944 \| 0.7796 \| 0.9741 \|
	\| 0.5374 \| 9000 \| 0.0050 \| 0.7773 \| 0.7844 \| 0.7808 \| 0.9747 \|
	\| 0.5672 \| 9500 \| 0.0049 \| 0.7954 \| 0.7711 \| 0.7830 \| 0.9757 \|
	\| 0.5971 \| 10000 \| 0.0049 \| 0.7844 \| 0.7876 \| 0.7860 \| 0.9754 \|
	\| 0.6270 \| 10500 \| 0.0047 \| 0.7898 \| 0.7940 \| 0.7919 \| 0.9761 \|
	\| 0.6568 \| 11000 \| 0.0047 \| 0.7852 \| 0.7929 \| 0.7890 \| 0.9761 \|
	\| 0.6867 \| 11500 \| 0.0047 \| 0.8001 \| 0.7908 \| 0.7954 \| 0.9770 \|
	\| 0.7165 \| 12000 \| 0.0050 \| 0.7643 \| 0.8145 \| 0.7886 \| 0.9755 \|
	\| 0.7464 \| 12500 \| 0.0047 \| 0.7991 \| 0.7892 \| 0.7941 \| 0.9764 \|
	\| 0.7762 \| 13000 \| 0.0046 \| 0.7948 \| 0.8084 \| 0.8015 \| 0.9774 \|
	\| 0.8061 \| 13500 \| 0.0046 \| 0.7841 \| 0.8154 \| 0.7994 \| 0.9771 \|
	\| 0.8359 \| 14000 \| 0.0043 \| 0.8283 \| 0.7776 \| 0.8021 \| 0.9783 \|
	\| 0.8658 \| 14500 \| 0.0044 \| 0.8054 \| 0.7993 \| 0.8023 \| 0.9773 \|
	\| 0.8957 \| 15000 \| 0.0047 \| 0.7704 \| 0.8152 \| 0.7922 \| 0.9758 \|
	\| 0.9255 \| 15500 \| 0.0043 \| 0.8018 \| 0.8149 \| 0.8083 \| 0.9782 \|
	\| 0.9554 \| 16000 \| 0.0043 \| 0.8255 \| 0.7938 \| 0.8093 \| 0.9789 \|
	\| 0.9852 \| 16500 \| 0.0042 \| 0.8201 \| 0.8008 \| 0.8104 \| 0.9787 \|
	\| 1.0151 \| 17000 \| 0.0044 \| 0.7947 \| 0.8175 \| 0.8059 \| 0.9784 \|
	\| 1.0449 \| 17500 \| 0.0044 \| 0.7942 \| 0.8195 \| 0.8066 \| 0.9777 \|
	\| 1.0748 \| 18000 \| 0.0043 \| 0.8124 \| 0.8110 \| 0.8117 \| 0.9789 \|
	\| 1.1046 \| 18500 \| 0.0043 \| 0.7987 \| 0.8157 \| 0.8071 \| 0.9788 \|
	\| 1.1345 \| 19000 \| 0.0043 \| 0.8037 \| 0.8171 \| 0.8103 \| 0.9789 \|
	\| 1.1644 \| 19500 \| 0.0042 \| 0.8178 \| 0.8076 \| 0.8127 \| 0.9796 \|
	\| 1.1942 \| 20000 \| 0.0044 \| 0.7803 \| 0.8389 \| 0.8085 \| 0.9780 \|
	\| 1.2241 \| 20500 \| 0.0043 \| 0.8040 \| 0.8210 \| 0.8124 \| 0.9790 \|
	\| 1.2539 \| 21000 \| 0.0043 \| 0.8038 \| 0.8245 \| 0.8141 \| 0.9788 \|
	\| 1.2838 \| 21500 \| 0.0041 \| 0.8318 \| 0.7973 \| 0.8142 \| 0.9794 \|
	\| 1.3136 \| 22000 \| 0.0041 \| 0.8106 \| 0.8211 \| 0.8158 \| 0.9796 \|
	\| 1.3435 \| 22500 \| 0.0041 \| 0.8288 \| 0.8046 \| 0.8165 \| 0.9796 \|
	\| 1.3733 \| 23000 \| 0.0041 \| 0.8218 \| 0.8170 \| 0.8194 \| 0.9799 \|
	\| 1.4032 \| 23500 \| 0.0042 \| 0.8164 \| 0.8171 \| 0.8168 \| 0.9799 \|
	\| 1.4330 \| 24000 \| 0.0041 \| 0.8105 \| 0.8248 \| 0.8176 \| 0.9793 \|
	\| 1.4629 \| 24500 \| 0.0042 \| 0.8073 \| 0.8196 \| 0.8134 \| 0.9791 \|
	\| 1.4928 \| 25000 \| 0.0040 \| 0.8211 \| 0.8162 \| 0.8187 \| 0.9797 \|
	\| 1.5226 \| 25500 \| 0.0040 \| 0.8195 \| 0.8225 \| 0.8210 \| 0.9800 \|
	\| 1.5525 \| 26000 \| 0.0040 \| 0.8372 \| 0.8018 \| 0.8191 \| 0.9799 \|
	\| 1.5823 \| 26500 \| 0.0040 \| 0.8263 \| 0.8161 \| 0.8212 \| 0.9802 \|
	\| 1.6122 \| 27000 \| 0.0039 \| 0.8275 \| 0.8141 \| 0.8208 \| 0.9802 \|
	\| 1.6420 \| 27500 \| 0.0040 \| 0.8264 \| 0.8198 \| 0.8231 \| 0.9804 \|
	\| 1.6719 \| 28000 \| 0.0040 \| 0.8218 \| 0.8195 \| 0.8206 \| 0.9799 \|
	\| 1.7017 \| 28500 \| 0.0039 \| 0.8286 \| 0.8195 \| 0.8240 \| 0.9803 \|
	\| 1.7316 \| 29000 \| 0.0041 \| 0.8004 \| 0.8357 \| 0.8177 \| 0.9788 \|
	\| 1.7615 \| 29500 \| 0.0040 \| 0.8138 \| 0.8304 \| 0.8220 \| 0.9801 \|
	\| 1.7913 \| 30000 \| 0.0040 \| 0.8160 \| 0.8309 \| 0.8234 \| 0.9804 \|
	\| 1.8212 \| 30500 \| 0.0039 \| 0.8204 \| 0.8262 \| 0.8233 \| 0.9802 \|
	\| 1.8510 \| 31000 \| 0.0038 \| 0.8292 \| 0.8228 \| 0.8260 \| 0.9810 \|
	\| 1.8809 \| 31500 \| 0.0039 \| 0.8247 \| 0.8246 \| 0.8246 \| 0.9806 \|
	\| 1.9107 \| 32000 \| 0.0038 \| 0.8267 \| 0.8258 \| 0.8262 \| 0.9810 \|
	\| 1.9406 \| 32500 \| 0.0039 \| 0.8102 \| 0.8398 \| 0.8248 \| 0.9805 \|
	\| 1.9704 \| 33000 \| 0.0039 \| 0.8321 \| 0.8185 \| 0.8253 \| 0.9809 \|
	\| 2.0003 \| 33500 \| 0.0038 \| 0.8325 \| 0.8261 \| 0.8293 \| 0.9814 \|
	\| 2.0302 \| 34000 \| 0.0038 \| 0.8352 \| 0.8228 \| 0.8289 \| 0.9813 \|
	\| 2.0600 \| 34500 \| 0.0041 \| 0.8144 \| 0.8369 \| 0.8255 \| 0.9809 \|
	\| 2.0899 \| 35000 \| 0.0039 \| 0.8274 \| 0.8281 \| 0.8277 \| 0.9813 \|
	\| 2.1197 \| 35500 \| 0.0039 \| 0.8198 \| 0.8353 \| 0.8275 \| 0.9812 \|
	\| 2.1496 \| 36000 \| 0.0039 \| 0.8211 \| 0.8358 \| 0.8284 \| 0.9811 \|
	\| 2.1794 \| 36500 \| 0.0039 \| 0.8242 \| 0.8300 \| 0.8271 \| 0.9809 \|
	\| 2.2093 \| 37000 \| 0.0039 \| 0.8194 \| 0.8317 \| 0.8255 \| 0.9808 \|
	\| 2.2391 \| 37500 \| 0.0039 \| 0.8258 \| 0.8344 \| 0.8301 \| 0.9814 \|
	\| 2.2690 \| 38000 \| 0.0039 \| 0.8292 \| 0.8302 \| 0.8297 \| 0.9816 \|
	\| 2.2989 \| 38500 \| 0.0039 \| 0.8281 \| 0.8315 \| 0.8298 \| 0.9813 \|
	\| 2.3287 \| 39000 \| 0.0039 \| 0.8174 \| 0.8386 \| 0.8279 \| 0.9808 \|
	\| 2.3586 \| 39500 \| 0.0039 \| 0.8208 \| 0.8364 \| 0.8285 \| 0.9810 \|
	\| 2.3884 \| 40000 \| 0.0039 \| 0.8230 \| 0.8379 \| 0.8304 \| 0.9815 \|
	\| 2.4183 \| 40500 \| 0.0038 \| 0.8355 \| 0.8273 \| 0.8314 \| 0.9816 \|
	\| 2.4481 \| 41000 \| 0.0038 \| 0.8290 \| 0.8347 \| 0.8319 \| 0.9816 \|
	\| 2.4780 \| 41500 \| 0.0038 \| 0.8233 \| 0.8403 \| 0.8317 \| 0.9815 \|
	\| 2.5078 \| 42000 \| 0.0039 \| 0.8186 \| 0.8417 \| 0.8300 \| 0.9814 \|
	\| 2.5377 \| 42500 \| 0.0038 \| 0.8321 \| 0.8343 \| 0.8332 \| 0.9818 \|
	\| 2.5675 \| 43000 \| 0.0038 \| 0.8239 \| 0.8396 \| 0.8317 \| 0.9816 \|
	\| 2.5974 \| 43500 \| 0.0038 \| 0.8267 \| 0.8378 \| 0.8322 \| 0.9816 \|
	\| 2.6273 \| 44000 \| 0.0038 \| 0.8325 \| 0.8343 \| 0.8334 \| 0.9818 \|
	\| 2.6571 \| 44500 \| 0.0038 \| 0.8254 \| 0.8399 \| 0.8326 \| 0.9817 \|
	\| 2.6870 \| 45000 \| 0.0038 \| 0.8339 \| 0.8338 \| 0.8339 \| 0.9820 \|
	\| 2.7168 \| 45500 \| 0.0038 \| 0.8301 \| 0.8381 \| 0.8341 \| 0.9819 \|
	\| 2.7467 \| 46000 \| 0.0038 \| 0.8309 \| 0.8371 \| 0.8340 \| 0.9818 \|
	\| 2.7765 \| 46500 \| 0.0038 \| 0.8296 \| 0.8377 \| 0.8337 \| 0.9817 \|
	\| 2.8064 \| 47000 \| 0.0037 \| 0.8337 \| 0.8349 \| 0.8343 \| 0.9820 \|
	\| 2.8362 \| 47500 \| 0.0037 \| 0.8303 \| 0.8387 \| 0.8345 \| 0.9820 \|
	\| 2.8661 \| 48000 \| 0.0037 \| 0.8289 \| 0.8401 \| 0.8344 \| 0.9819 \|
	\| 2.8960 \| 48500 \| 0.0037 \| 0.8299 \| 0.8400 \| 0.8349 \| 0.9820 \|
	\| 2.9258 \| 49000 \| 0.0037 \| 0.8289 \| 0.8401 \| 0.8344 \| 0.9819 \|
	\| 2.9557 \| 49500 \| 0.0037 \| 0.8322 \| 0.8380 \| 0.8351 \| 0.9821 \|
	\| 2.9855 \| 50000 \| 0.0037 \| 0.8312 \| 0.8384 \| 0.8348 \| 0.9820 \|

	### Framework Versions
	- Python: 3.11.7
	- SpanMarker: 1.5.0
	- Transformers: 4.42.1
	- PyTorch: 2.1.1+cu121
	- Datasets: 2.14.5
	- Tokenizers: 0.19.1

	## Citation

	### BibTeX
	```
	@software{Aarsen_SpanMarker,
	author = {Aarsen, Tom},
	license = {Apache-2.0},
	title = {{SpanMarker for Named Entity Recognition}},
	url = {https://github.com/tomaarsen/SpanMarkerNER}
	}
	```

	<!--
	## Glossary

	Clearly define terms in order to be accessible across audiences.
	-->

	<!--
	## Model Card Authors

	Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.
	-->

	<!--
	## Model Card Contact

	Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.
	-->