modelofinenew

This model is a fine-tuned version of projecte-aina/roberta-base-ca-v2-cased-te on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.2185
• Accuracy: 0.5126
• F1: 0.5338

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 20
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.9546	1.6129	50	2.5028	0.2101	0.2019
1.7901	3.2258	100	2.6787	0.1849	0.1805
1.6177	4.8387	150	2.3416	0.3445	0.3332
1.2977	6.4516	200	2.0729	0.4202	0.4060
0.9411	8.0645	250	1.9746	0.4706	0.4583
0.595	9.6774	300	1.8840	0.5126	0.5167
0.3374	11.2903	350	1.8955	0.4958	0.4977
0.1974	12.9032	400	1.9658	0.5378	0.5169
0.0981	14.5161	450	2.2185	0.5126	0.5338
0.05	16.1290	500	2.3554	0.5042	0.5096
0.0312	17.7419	550	2.4366	0.5294	0.5289
0.0235	19.3548	600	2.5235	0.5210	0.5181
0.0194	20.9677	650	2.5713	0.5294	0.5289
0.0166	22.5806	700	2.6188	0.5294	0.5289
0.0148	24.1935	750	2.6473	0.5294	0.5289
0.0136	25.8065	800	2.6742	0.5210	0.5218
0.013	27.4194	850	2.6920	0.5210	0.5218
0.0129	29.0323	900	2.6961	0.5210	0.5218

Framework versions

• Transformers 4.40.0
• Pytorch 2.1.0+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1