scenario-NON-KD-PR-COPY-D2_data-AmazonScience_massive_all_1_1_alpha-jason

This model is a fine-tuned version of xlm-roberta-base on the massive dataset. It achieves the following results on the evaluation set:

• Loss: 1.5020
• Accuracy: 0.8325
• F1: 0.8079

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 111
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.6751	0.27	5000	1.5417	0.5730	0.4210
1.187	0.53	10000	1.1380	0.6887	0.5642
0.9866	0.8	15000	0.9834	0.7333	0.6460
0.8094	1.07	20000	0.8827	0.7621	0.6773
0.7239	1.34	25000	0.8257	0.7791	0.7096
0.6985	1.6	30000	0.7914	0.7915	0.7423
0.6357	1.87	35000	0.7729	0.7978	0.7488
0.5143	2.14	40000	0.7910	0.7985	0.7496
0.5058	2.41	45000	0.7437	0.8084	0.7662
0.4965	2.67	50000	0.7472	0.8105	0.7631
0.4794	2.94	55000	0.7226	0.8152	0.7781
0.3976	3.21	60000	0.7433	0.8172	0.7819
0.3856	3.47	65000	0.7671	0.8133	0.7765
0.3822	3.74	70000	0.7587	0.8169	0.7788
0.3535	4.01	75000	0.7766	0.8176	0.7793
0.3031	4.28	80000	0.7828	0.8217	0.7883
0.3171	4.54	85000	0.7896	0.8200	0.7807
0.3023	4.81	90000	0.7956	0.8207	0.7884
0.2466	5.08	95000	0.8068	0.8257	0.7915
0.2562	5.34	100000	0.8279	0.8234	0.7871
0.256	5.61	105000	0.8033	0.8251	0.7888
0.2552	5.88	110000	0.8100	0.8244	0.7931
0.2067	6.15	115000	0.8484	0.8244	0.7999
0.2106	6.41	120000	0.8735	0.8209	0.7902
0.2162	6.68	125000	0.8604	0.8248	0.7979
0.2065	6.95	130000	0.8785	0.8252	0.7994
0.1734	7.22	135000	0.9147	0.8263	0.7984
0.1801	7.48	140000	0.9060	0.8265	0.8021
0.1724	7.75	145000	0.8960	0.8281	0.8001
0.1496	8.02	150000	0.9022	0.8279	0.8005
0.1424	8.28	155000	0.9477	0.8260	0.8021
0.1585	8.55	160000	0.9441	0.8269	0.8012
0.1551	8.82	165000	0.9448	0.8270	0.8022
0.1176	9.09	170000	0.9803	0.8291	0.8041
0.1219	9.35	175000	0.9954	0.8273	0.8002
0.1292	9.62	180000	1.0029	0.8247	0.7988
0.1312	9.89	185000	1.0175	0.8234	0.7972
0.0974	10.15	190000	1.0531	0.8273	0.8000
0.1072	10.42	195000	1.0572	0.8257	0.8006
0.1247	10.69	200000	1.0197	0.8283	0.8044
0.1093	10.96	205000	1.0303	0.8283	0.8042
0.0951	11.22	210000	1.0860	0.8275	0.8001
0.1027	11.49	215000	1.0792	0.8275	0.8020
0.0998	11.76	220000	1.0733	0.8292	0.8021
0.0747	12.03	225000	1.1062	0.8280	0.8026
0.0834	12.29	230000	1.1240	0.8287	0.8040
0.0823	12.56	235000	1.1432	0.8277	0.8028
0.0837	12.83	240000	1.1317	0.8287	0.8026
0.0669	13.09	245000	1.1684	0.8261	0.7982
0.0738	13.36	250000	1.1888	0.8268	0.7988
0.0757	13.63	255000	1.2008	0.8269	0.8004
0.0843	13.9	260000	1.1889	0.8260	0.7972
0.0639	14.16	265000	1.2192	0.8268	0.8045
0.0633	14.43	270000	1.2457	0.8268	0.8023
0.0637	14.7	275000	1.2309	0.8284	0.8029
0.0752	14.96	280000	1.2529	0.8244	0.8009
0.0611	15.23	285000	1.2513	0.8320	0.8061
0.0636	15.5	290000	1.2538	0.8280	0.8017
0.0568	15.77	295000	1.2798	0.8290	0.8020
0.0426	16.03	300000	1.2813	0.8274	0.8020
0.0498	16.3	305000	1.2822	0.8290	0.8070
0.0588	16.57	310000	1.2883	0.8295	0.8037
0.0556	16.84	315000	1.2992	0.8285	0.8018
0.0405	17.1	320000	1.3210	0.8292	0.8033
0.0452	17.37	325000	1.3339	0.8288	0.8035
0.0468	17.64	330000	1.3359	0.8272	0.8033
0.051	17.9	335000	1.3433	0.8272	0.8022
0.0333	18.17	340000	1.3592	0.8287	0.8013
0.0419	18.44	345000	1.3444	0.8296	0.8058
0.0496	18.71	350000	1.3343	0.8300	0.8064
0.0449	18.97	355000	1.3581	0.8285	0.8023
0.0362	19.24	360000	1.3908	0.8280	0.8030
0.0357	19.51	365000	1.3933	0.8278	0.8035
0.0343	19.77	370000	1.3751	0.8297	0.8041
0.0284	20.04	375000	1.3843	0.8301	0.8046
0.0341	20.31	380000	1.4052	0.8287	0.8051
0.0373	20.58	385000	1.4041	0.8288	0.8029
0.0325	20.84	390000	1.4135	0.8299	0.8049
0.0232	21.11	395000	1.4253	0.8297	0.8061
0.0285	21.38	400000	1.4292	0.8301	0.8068
0.0319	21.65	405000	1.4289	0.8308	0.8068
0.0279	21.91	410000	1.4378	0.8287	0.8064
0.0225	22.18	415000	1.4452	0.8310	0.8058
0.0327	22.45	420000	1.4419	0.8301	0.8062
0.0295	22.71	425000	1.4375	0.8315	0.8066
0.0277	22.98	430000	1.4604	0.8302	0.8073
0.0262	23.25	435000	1.4549	0.8316	0.8070
0.0259	23.52	440000	1.4527	0.8321	0.8076
0.0281	23.78	445000	1.4669	0.8303	0.8057
0.0216	24.05	450000	1.4659	0.8305	0.8063
0.0214	24.32	455000	1.4814	0.8304	0.8057
0.0239	24.58	460000	1.4540	0.8315	0.8060
0.0228	24.85	465000	1.4743	0.8308	0.8061
0.0232	25.12	470000	1.4859	0.8302	0.8049
0.0196	25.39	475000	1.4926	0.8302	0.8049
0.0193	25.65	480000	1.4848	0.8317	0.8064
0.0235	25.92	485000	1.4784	0.8313	0.8075
0.0177	26.19	490000	1.4941	0.8308	0.8069
0.0165	26.46	495000	1.5015	0.8322	0.8083
0.0183	26.72	500000	1.4910	0.8311	0.8073
0.0193	26.99	505000	1.4894	0.8314	0.8083
0.0151	27.26	510000	1.4850	0.8321	0.8068
0.0151	27.52	515000	1.4990	0.8319	0.8072
0.0155	27.79	520000	1.4953	0.8321	0.8071
0.014	28.06	525000	1.4994	0.8318	0.8079
0.0151	28.33	530000	1.4997	0.8310	0.8058
0.0164	28.59	535000	1.4911	0.8315	0.8067
0.0168	28.86	540000	1.4924	0.8324	0.8082
0.0123	29.13	545000	1.5005	0.8318	0.8076
0.013	29.39	550000	1.5007	0.8321	0.8076
0.0147	29.66	555000	1.5022	0.8325	0.8076
0.0138	29.93	560000	1.5020	0.8325	0.8079

Framework versions

• Transformers 4.33.3
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3