RoBERTa_Sentiment_Analysis

This model is a fine-tuned version of roberta-base on Twitter Sentiment Analysis dataset

It achieves the following results on the evaluation set:

• Loss: 0.0994

Model description

Fine-tuning performed on a pretrained RoBERTa model. The code can be found here

Intended uses & limitations

The model is used to classify tweets as either being neutral or hate speech

'test.csv' of Twitter Sentiment Analysis is unused and unlabelled dataset. Contributions in code to utilize the dataset for evaluation are welcome!

Training and evaluation data

'train.csv' of Twitter Sentiment Analysis is split into training and evaluation sets (80-20)

Fine-tuning was carried out on Google Colab's T4 GPU

Training procedure

RobertaTokenizerFast is used for tokenizing preprocessed data

Pretrained RobertaForSequenceClassification is used as the classification model

Hyperparameters are defined in TrainingArguments and Trainer is used to train the model

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 50
• eval_batch_size: 50
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 5
• weight_decay : 0.0000001
• report_to="tensorboard"

Training results

Training Loss	Epoch	Step	Validation Loss
0.1276	1.0	512	0.1116
0.1097	2.0	1024	0.0994
0.0662	3.0	1536	0.1165
0.0542	4.0	2048	0.1447
0.019	5.0	2560	0.1630

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1