Distilbert-base-uncased-emotion

Model description:

Distilbert is created with knowledge distillation during the pre-training phase which reduces the size of a BERT model by 40%, while retaining 97% of its language understanding. It's smaller, faster than Bert and any other Bert-based model.

Distilbert-base-uncased finetuned on the emotion dataset using HuggingFace Trainer with below Hyperparameters

 learning rate 2e-5, 
 batch size 64,
 num_train_epochs=8,

Model Performance Comparision on Emotion Dataset from Twitter:

Model Accuracy F1 Score Test Sample per Second
Distilbert-base-uncased-emotion 93.8 93.79 398.69
Bert-base-uncased-emotion 94.05 94.06 190.152
Roberta-base-emotion 93.95 93.97 195.639
Albert-base-v2-emotion 93.6 93.65 182.794

How to Use the model:

from transformers import pipeline
classifier = pipeline("text-classification",model='bhadresh-savani/distilbert-base-uncased-emotion', return_all_scores=True)
prediction = classifier("I love using transformers. The best part is wide range of support and its easy to use", )
print(prediction)

"""
Output:
[[
{'label': 'sadness', 'score': 0.0006792712374590337}, 
{'label': 'joy', 'score': 0.9959300756454468}, 
{'label': 'love', 'score': 0.0009452480007894337}, 
{'label': 'anger', 'score': 0.0018055217806249857}, 
{'label': 'fear', 'score': 0.00041110432357527316}, 
{'label': 'surprise', 'score': 0.0002288572577526793}
]]
"""

Dataset:

Twitter-Sentiment-Analysis.

Training procedure

Colab Notebook

Eval results

{
'test_accuracy': 0.938,
 'test_f1': 0.937932884041714,
 'test_loss': 0.1472451239824295,
 'test_mem_cpu_alloc_delta': 0,
 'test_mem_cpu_peaked_delta': 0,
 'test_mem_gpu_alloc_delta': 0,
 'test_mem_gpu_peaked_delta': 163454464,
 'test_runtime': 5.0164,
 'test_samples_per_second': 398.69
 }

Reference:

Downloads last month
48,772
Safetensors
Model size
67M params
Tensor type
F32
Β·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for bhadresh-savani/distilbert-base-uncased-emotion

Finetunes
2 models

Dataset used to train bhadresh-savani/distilbert-base-uncased-emotion

Spaces using bhadresh-savani/distilbert-base-uncased-emotion 39

Evaluation results