|
--- |
|
license: apache-2.0 |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- AdamCodd/emotion-balanced |
|
metrics: |
|
- accuracy |
|
- f1 |
|
- recall |
|
- precision |
|
base_model: bert-tiny |
|
model-index: |
|
- name: tinybert-emotion-balanced |
|
results: |
|
- task: |
|
type: text-classification |
|
name: Text Classification |
|
dataset: |
|
name: emotion |
|
type: emotion |
|
args: default |
|
metrics: |
|
- type: accuracy |
|
value: 0.9354 |
|
name: Accuracy |
|
- type: loss |
|
value: 0.1809 |
|
name: Loss |
|
- type: f1 |
|
value: 0.9354946613311768 |
|
name: F1 |
|
--- |
|
|
|
# tinybert-emotion |
|
|
|
This model is a fine-tuned version of [bert-tiny](https://huggingface.co/prajjwal1/bert-tiny) on the [emotion balanced dataset](https://huggingface.co/datasets/AdamCodd/emotion-balanced). |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.1809 |
|
- Accuracy: 0.9354 |
|
|
|
## Model description |
|
|
|
TinyBERT is 7.5 times smaller and 9.4 times faster on inference compared to its teacher BERT model (while DistilBERT is 40% smaller and 1.6 times faster than BERT). |
|
The model has been trained on 89_754 examples split into train, validation and test. Each label was perfectly balanced in each split. |
|
|
|
## Intended uses & limitations |
|
|
|
This model is not as accurate as the [distilbert-emotion-balanced](https://huggingface.co/AdamCodd/distilbert-base-uncased-finetuned-emotion-balanced) one because the focus was on speed, which can lead to misinterpretation of complex sentences. Despite this, its performance is quite good and should be more than sufficient for most use cases. |
|
|
|
Usage: |
|
```python |
|
from transformers import pipeline |
|
|
|
# Create the pipeline |
|
emotion_classifier = pipeline('text-classification', model='AdamCodd/tinybert-emotion-balanced') |
|
|
|
# Now you can use the pipeline to classify emotions |
|
result = emotion_classifier("We are delighted that you will be coming to visit us. It will be so nice to have you here.") |
|
print(result) |
|
#[{'label': 'joy', 'score': 0.9895486831665039}] |
|
``` |
|
This model faces challenges in accurately categorizing negative sentences, as well as those containing elements of sarcasm or irony. These limitations are largely attributable to TinyBERT's constrained capabilities in semantic understanding. Although the model is generally proficient in emotion detection tasks, it may lack the nuance necessary for interpreting complex emotional nuances. |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 3e-05 |
|
- train_batch_size: 32 |
|
- eval_batch_size: 64 |
|
- seed: 1270 |
|
- optimizer: AdamW with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 150 |
|
- num_epochs: 10 |
|
- weight_decay: 0.01 |
|
|
|
### Training results |
|
|
|
precision recall f1-score support |
|
|
|
sadness 0.9733 0.9245 0.9482 1496 |
|
joy 0.9651 0.8864 0.9240 1496 |
|
love 0.9127 0.9786 0.9445 1496 |
|
anger 0.9479 0.9365 0.9422 1496 |
|
fear 0.9213 0.9004 0.9108 1496 |
|
surprise 0.9016 0.9866 0.9422 1496 |
|
|
|
accuracy 0.9355 8976 |
|
macro avg 0.9370 0.9355 0.9353 8976 |
|
weighted avg 0.9370 0.9355 0.9353 8976 |
|
|
|
test_acc: 0.9354946613311768 |
|
test_loss: 0.1809326708316803 |
|
|
|
### Framework versions |
|
|
|
- Transformers 4.33.0 |
|
- Pytorch lightning 2.0.8 |
|
- Tokenizers 0.13.3 |
|
|
|
If you want to support me, you can [here](https://ko-fi.com/adamcodd). |
