predict-perception-bertino-focus-victim

This model is a fine-tuned version of indigo-ai/BERTino on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2497
• R2: 0.6131

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 20
• eval_batch_size: 8
• seed: 1996
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 47

Training results

Training Loss	Epoch	Step	Validation Loss	R2
0.5438	1.0	14	0.4405	0.3175
0.2336	2.0	28	0.2070	0.6792
0.0986	3.0	42	0.2868	0.5555
0.0907	4.0	56	0.2916	0.5481
0.0652	5.0	70	0.2187	0.6611
0.0591	6.0	84	0.2320	0.6406
0.0478	7.0	98	0.2501	0.6125
0.0347	8.0	112	0.2425	0.6243
0.021	9.0	126	0.2670	0.5863
0.0214	10.0	140	0.2853	0.5580
0.0172	11.0	154	0.2726	0.5776
0.0177	12.0	168	0.2629	0.5927
0.0152	13.0	182	0.2396	0.6287
0.012	14.0	196	0.2574	0.6012
0.0119	15.0	210	0.2396	0.6288
0.0128	16.0	224	0.2517	0.6100
0.0109	17.0	238	0.2509	0.6112
0.008	18.0	252	0.2522	0.6092
0.0101	19.0	266	0.2503	0.6121
0.0075	20.0	280	0.2527	0.6084
0.0082	21.0	294	0.2544	0.6058
0.0061	22.0	308	0.2510	0.6111
0.006	23.0	322	0.2402	0.6279
0.005	24.0	336	0.2539	0.6066
0.0058	25.0	350	0.2438	0.6222
0.0051	26.0	364	0.2439	0.6221
0.006	27.0	378	0.2442	0.6216
0.0061	28.0	392	0.2416	0.6257
0.0053	29.0	406	0.2519	0.6097
0.0045	30.0	420	0.2526	0.6085
0.0034	31.0	434	0.2578	0.6006
0.0039	32.0	448	0.2557	0.6038
0.0043	33.0	462	0.2538	0.6068
0.0041	34.0	476	0.2535	0.6072
0.0042	35.0	490	0.2560	0.6033
0.0037	36.0	504	0.2576	0.6009
0.0036	37.0	518	0.2634	0.5919
0.0037	38.0	532	0.2582	0.5999
0.0038	39.0	546	0.2552	0.6045
0.0034	40.0	560	0.2563	0.6028
0.0033	41.0	574	0.2510	0.6110
0.0029	42.0	588	0.2515	0.6103
0.0033	43.0	602	0.2525	0.6088
0.0028	44.0	616	0.2522	0.6093
0.0028	45.0	630	0.2526	0.6085
0.0027	46.0	644	0.2494	0.6136
0.0024	47.0	658	0.2497	0.6131

Framework versions

• Transformers 4.16.2
• Pytorch 1.10.2+cu113
• Datasets 1.18.3
• Tokenizers 0.11.0