twitter-roberta-base-sentiment-latest_12112024T150727

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-sentiment-latest on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3348
• F1: 0.4579
• Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 600
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Rate
No log	0.9942	86	1.8285	0.1193	0.0000
No log	2.0	173	1.8031	0.3302	0.0000
No log	2.9942	259	1.5578	0.3690	0.0000
No log	4.0	346	1.4611	0.4092	0.0000
No log	4.9942	432	1.4700	0.4079	0.0000
1.3786	6.0	519	1.3348	0.4579	0.0000
1.3786	6.9942	605	1.6543	0.4193	1e-05
1.3786	8.0	692	1.4421	0.4858	1e-05
1.3786	8.9942	778	1.5573	0.4603	0.0000
1.3786	10.0	865	1.5451	0.4797	0.0000
1.3786	10.9942	951	1.8338	0.4396	0.0000
0.6407	12.0	1038	1.9383	0.4364	0.0000
0.6407	12.9942	1124	1.7573	0.4680	0.0000
0.6407	14.0	1211	1.8321	0.4735	0.0000
0.6407	14.9942	1297	1.9524	0.4619	0.0000
0.6407	16.0	1384	2.1822	0.4591	0.0000
0.6407	16.9942	1470	2.1302	0.4686	6e-06
0.2608	18.0	1557	2.5139	0.4467	0.0000
0.2608	18.9942	1643	2.3385	0.4641	0.0000
0.2608	20.0	1730	2.3281	0.4726	0.0000
0.2608	20.9942	1816	2.5489	0.4722	0.0000
0.2608	22.0	1903	2.5727	0.4745	0.0000
0.2608	22.9942	1989	2.5584	0.4694	0.0000
0.1026	24.0	2076	2.8115	0.4584	0.0000
0.1026	24.9942	2162	2.7270	0.4691	0.0000
0.1026	26.0	2249	2.7379	0.4746	7e-07
0.1026	26.9942	2335	2.8336	0.4757	4e-07
0.1026	28.0	2422	2.8201	0.4703	2e-07
0.057	28.9942	2508	2.8292	0.4691	0.0
0.057	29.8266	2580	2.8271	0.4691	0.0

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.19.1