File size: 6,860 Bytes
f6b8a39 61e4faf 4dbb2ab f6b8a39 2b8bf11 1aedbd3 842a565 1aedbd3 2b8bf11 1aedbd3 f6b8a39 61e4faf f6b8a39 842a565 f6b8a39 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 |
---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- private
base_model: t5-large
model-index:
- name: ner-news-t5-large
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# T5-Encoder(T5-large model) fine-tuned on very small dataset for token classification
Simple experimental model that was trained in 3 epochs on very small dataset
## Usage
```python
from transformers import AutoTokenizer, AutoModelForTokenClassification, NerPipeline
model = AutoModelForTokenClassification.from_pretrained("imvladikon/t5-english-ner", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("imvladikon/t5-english-ner", trust_remote_code=True)
pipe = NerPipeline(model=model, tokenizer=tokenizer, aggregation_strategy="max")
print(pipe("London is the capital city of England and the United Kingdom"))
"""
[{'entity_group': 'LOCATION',
'score': 0.84536326,
'word': 'London',
'start': 0,
'end': 6},
{'entity_group': 'LOCATION',
'score': 0.8957489,
'word': 'England',
'start': 30,
'end': 37},
{'entity_group': 'LOCATION',
'score': 0.73186326,
'word': 'UnitedKingdom',
'start': 46,
'end': 60}]
"""
```
## Usage in spacy
```bash
pip install spacy transformers git+https://github.com/explosion/spacy-huggingface-pipelines -q
```
```python
import spacy
from spacy import displacy
text = "My name is Sarah and I live in London"
nlp = spacy.blank("en")
nlp.add_pipe("hf_token_pipe", config={"model": "imvladikon/t5-english-ner", "kwargs": {"trust_remote_code":True}})
doc = nlp(text)
print(doc.ents)
# (Sarah, London)
```
This model is a fine-tuned version of [t5-large](https://huggingface.co/t5-large) on the private(en) dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1956
- Commercial Item Precision: 0.0
- Commercial Item Recall: 0.0
- Commercial Item F1: 0.0
- Commercial Item Number: 1
- Date Precision: 0.8125
- Date Recall: 0.9286
- Date F1: 0.8667
- Date Number: 14
- Location Precision: 0.7143
- Location Recall: 0.75
- Location F1: 0.7317
- Location Number: 20
- Organization Precision: 0.8588
- Organization Recall: 0.9125
- Organization F1: 0.8848
- Organization Number: 80
- Other Precision: 0.3684
- Other Recall: 0.3333
- Other F1: 0.35
- Other Number: 21
- Person Precision: 0.8182
- Person Recall: 0.9310
- Person F1: 0.8710
- Person Number: 29
- Quantity Precision: 0.8
- Quantity Recall: 0.8571
- Quantity F1: 0.8276
- Quantity Number: 14
- Title Precision: 0.0
- Title Recall: 0.0
- Title F1: 0.0
- Title Number: 7
- Overall Precision: 0.75
- Overall Recall: 0.7903
- Overall F1: 0.7696
- Overall Accuracy: 0.9534
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0
### Training results
| Training Loss | Epoch | Step | Validation Loss | Commercial Item Precision | Commercial Item Recall | Commercial Item F1 | Commercial Item Number | Date Precision | Date Recall | Date F1 | Date Number | Location Precision | Location Recall | Location F1 | Location Number | Organization Precision | Organization Recall | Organization F1 | Organization Number | Other Precision | Other Recall | Other F1 | Other Number | Person Precision | Person Recall | Person F1 | Person Number | Quantity Precision | Quantity Recall | Quantity F1 | Quantity Number | Title Precision | Title Recall | Title F1 | Title Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------:|:----------------------:|:------------------:|:----------------------:|:--------------:|:-----------:|:-------:|:-----------:|:------------------:|:---------------:|:-----------:|:---------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:---------------:|:------------:|:--------:|:------------:|:----------------:|:-------------:|:---------:|:-------------:|:------------------:|:---------------:|:-----------:|:---------------:|:---------------:|:------------:|:--------:|:------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 0.8868 | 1.0 | 708 | 0.2725 | 0.0 | 0.0 | 0.0 | 1 | 0.8125 | 0.9286 | 0.8667 | 14 | 0.4167 | 0.75 | 0.5357 | 20 | 0.8272 | 0.8375 | 0.8323 | 80 | 1.0 | 0.0476 | 0.0909 | 21 | 0.8438 | 0.9310 | 0.8852 | 29 | 0.6667 | 0.7143 | 0.6897 | 14 | 0.0 | 0.0 | 0.0 | 7 | 0.7348 | 0.7151 | 0.7248 | 0.9446 |
| 0.2984 | 2.0 | 1416 | 0.2121 | 0.0 | 0.0 | 0.0 | 1 | 0.8667 | 0.9286 | 0.8966 | 14 | 0.5 | 0.8 | 0.6154 | 20 | 0.8375 | 0.8375 | 0.8375 | 80 | 0.3077 | 0.1905 | 0.2353 | 21 | 0.8182 | 0.9310 | 0.8710 | 29 | 0.7333 | 0.7857 | 0.7586 | 14 | 0.0 | 0.0 | 0.0 | 7 | 0.7077 | 0.7419 | 0.7244 | 0.9481 |
| 0.1729 | 3.0 | 2124 | 0.1956 | 0.0 | 0.0 | 0.0 | 1 | 0.8125 | 0.9286 | 0.8667 | 14 | 0.7143 | 0.75 | 0.7317 | 20 | 0.8588 | 0.9125 | 0.8848 | 80 | 0.3684 | 0.3333 | 0.35 | 21 | 0.8182 | 0.9310 | 0.8710 | 29 | 0.8 | 0.8571 | 0.8276 | 14 | 0.0 | 0.0 | 0.0 | 7 | 0.75 | 0.7903 | 0.7696 | 0.9534 |
### Framework versions
- Transformers 4.21.1
- Pytorch 1.12.0+cu113
- Datasets 2.4.0
- Tokenizers 0.12.1
## WANDB
[training logs and reports](https://wandb.ai/imvladikon/huggingface/runs/uyl6ihl1) |