2020-Q2-full_tweets_combined90

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9422

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.02	8000	2.2220
2.4154	0.03	16000	2.1427
2.4154	0.05	24000	2.1028
2.2273	0.07	32000	2.0824
2.2273	0.08	40000	2.0645
2.1774	0.1	48000	2.0478
2.1774	0.12	56000	2.0327
2.1569	0.13	64000	2.0248
2.1569	0.15	72000	2.0209
2.1439	0.17	80000	2.0049
2.1439	0.19	88000	2.0113
2.1271	0.2	96000	2.0038
2.1271	0.22	104000	2.0065
2.1211	0.24	112000	1.9987
2.1211	0.25	120000	1.9929
2.1194	0.27	128000	1.9922
2.1194	0.29	136000	1.9917
2.1118	0.3	144000	1.9885
2.1118	0.32	152000	1.9870
2.1047	0.34	160000	1.9843
2.1047	0.35	168000	1.9827
2.1015	0.37	176000	1.9826
2.1015	0.39	184000	1.9774
2.1042	0.4	192000	1.9771
2.1042	0.42	200000	1.9770
2.0919	0.44	208000	1.9752
2.0919	0.45	216000	1.9775
2.0953	0.47	224000	1.9684
2.0953	0.49	232000	1.9748
2.0848	0.51	240000	1.9714
2.0848	0.52	248000	1.9781
2.0882	0.54	256000	1.9709
2.0882	0.56	264000	1.9660
2.0922	0.57	272000	1.9651
2.0922	0.59	280000	1.9678
2.0938	0.61	288000	1.9667
2.0938	0.62	296000	1.9630
2.095	0.64	304000	1.9642
2.095	0.66	312000	1.9624
2.0908	0.67	320000	1.9603
2.0908	0.69	328000	1.9649
2.0927	0.71	336000	1.9641
2.0927	0.72	344000	1.9603
2.0931	0.74	352000	1.9590
2.0931	0.76	360000	1.9644
2.087	0.77	368000	1.9635
2.087	0.79	376000	1.9614
2.0792	0.81	384000	1.9591
2.0792	0.83	392000	1.9575
2.0899	0.84	400000	1.9592
2.0899	0.86	408000	1.9619
2.0812	0.88	416000	1.9582
2.0812	0.89	424000	1.9580
2.0948	0.91	432000	1.9587
2.0948	0.93	440000	1.9593
2.0895	0.94	448000	1.9608
2.0895	0.96	456000	1.9566
2.0756	0.98	464000	1.9525
2.0756	0.99	472000	1.9541
2.0842	1.01	480000	1.9601
2.0842	1.03	488000	1.9564
2.0935	1.04	496000	1.9522
2.0935	1.06	504000	1.9532
2.0836	1.08	512000	1.9537
2.0836	1.09	520000	1.9553
2.0876	1.11	528000	1.9469
2.0876	1.13	536000	1.9497
2.0778	1.15	544000	1.9542
2.0778	1.16	552000	1.9516
2.0829	1.18	560000	1.9506
2.0829	1.2	568000	1.9505
2.0864	1.21	576000	1.9531
2.0864	1.23	584000	1.9455
2.0893	1.25	592000	1.9471
2.0893	1.26	600000	1.9539
2.0808	1.28	608000	1.9455
2.0808	1.3	616000	1.9497
2.0838	1.31	624000	1.9466
2.0838	1.33	632000	1.9498
2.0812	1.35	640000	1.9510
2.0812	1.36	648000	1.9526
2.0793	1.38	656000	1.9471
2.0793	1.4	664000	1.9469
2.0789	1.41	672000	1.9455
2.0789	1.43	680000	1.9469
2.0883	1.45	688000	1.9439
2.0883	1.47	696000	1.9439
2.09	1.48	704000	1.9416
2.09	1.5	712000	1.9492
2.0845	1.52	720000	1.9430
2.0845	1.53	728000	1.9484
2.0742	1.55	736000	1.9456
2.0742	1.57	744000	1.9380
2.0839	1.58	752000	1.9418
2.0839	1.6	760000	1.9434
2.0806	1.62	768000	1.9450
2.0806	1.63	776000	1.9426
2.0805	1.65	784000	1.9441
2.0805	1.67	792000	1.9459
2.0833	1.68	800000	1.9435
2.0833	1.7	808000	1.9455
2.0763	1.72	816000	1.9421
2.0763	1.73	824000	1.9438
2.0758	1.75	832000	1.9371
2.0758	1.77	840000	1.9432
2.0888	1.79	848000	1.9414
2.0888	1.8	856000	1.9444
2.0786	1.82	864000	1.9408
2.0786	1.84	872000	1.9397
2.079	1.85	880000	1.9406
2.079	1.87	888000	1.9442
2.0817	1.89	896000	1.9404
2.0817	1.9	904000	1.9450
2.0792	1.92	912000	1.9380
2.0792	1.94	920000	1.9385
2.0741	1.95	928000	1.9449
2.0741	1.97	936000	1.9414
2.0832	1.99	944000	1.9402
2.0832	2.0	952000	1.9410
2.0695	2.02	960000	1.9371
2.0695	2.04	968000	1.9342
2.0813	2.05	976000	1.9376
2.0813	2.07	984000	1.9397
2.0804	2.09	992000	1.9394
2.0804	2.11	1000000	1.9370
2.0789	2.12	1008000	1.9350
2.0789	2.14	1016000	1.9327
2.0754	2.16	1024000	1.9421
2.0754	2.17	1032000	1.9371
2.0774	2.19	1040000	1.9411
2.0774	2.21	1048000	1.9337
2.0766	2.22	1056000	1.9387
2.0766	2.24	1064000	1.9334
2.079	2.26	1072000	1.9386
2.079	2.27	1080000	1.9335
2.068	2.29	1088000	1.9363
2.068	2.31	1096000	1.9420
2.0786	2.32	1104000	1.9331
2.0786	2.34	1112000	1.9327
2.0734	2.36	1120000	1.9391
2.0734	2.37	1128000	1.9363
2.0787	2.39	1136000	1.9321
2.0787	2.41	1144000	1.9333
2.0731	2.43	1152000	1.9369
2.0731	2.44	1160000	1.9357
2.0816	2.46	1168000	1.9353
2.0816	2.48	1176000	1.9319
2.0758	2.49	1184000	1.9366
2.0758	2.51	1192000	1.9301
2.0725	2.53	1200000	1.9329
2.0725	2.54	1208000	1.9370
2.085	2.56	1216000	1.9251
2.085	2.58	1224000	1.9369
2.0809	2.59	1232000	1.9377
2.0809	2.61	1240000	1.9398
2.0742	2.63	1248000	1.9368
2.0742	2.64	1256000	1.9389
2.0743	2.66	1264000	1.9287
2.0743	2.68	1272000	1.9337
2.0822	2.69	1280000	1.9323
2.0822	2.71	1288000	1.9348
2.0845	2.73	1296000	1.9328
2.0845	2.75	1304000	1.9324
2.0706	2.76	1312000	1.9304
2.0706	2.78	1320000	1.9322
2.0813	2.8	1328000	1.9320
2.0813	2.81	1336000	1.9379
2.0768	2.83	1344000	1.9283
2.0768	2.85	1352000	1.9352
2.0776	2.86	1360000	1.9266
2.0776	2.88	1368000	1.9339
2.0776	2.9	1376000	1.9371
2.0776	2.91	1384000	1.9353
2.072	2.93	1392000	1.9290
2.072	2.95	1400000	1.9337
2.077	2.96	1408000	1.9318
2.077	2.98	1416000	1.9326
2.0777	3.0	1424000	1.9338
2.0777	3.01	1432000	1.9307
2.0846	3.03	1440000	1.9305
2.0846	3.05	1448000	1.9312
2.0744	3.07	1456000	1.9332
2.0744	3.08	1464000	1.9313
2.0767	3.1	1472000	1.9311
2.0767	3.12	1480000	1.9322
2.082	3.13	1488000	1.9362
2.082	3.15	1496000	1.9329
2.0774	3.17	1504000	1.9335
2.0774	3.18	1512000	1.9342
2.0793	3.2	1520000	1.9326
2.0793	3.22	1528000	1.9313
2.0834	3.23	1536000	1.9302
2.0834	3.25	1544000	1.9299
2.0698	3.27	1552000	1.9288
2.0698	3.28	1560000	1.9311
2.0721	3.3	1568000	1.9262
2.0721	3.32	1576000	1.9320
2.0742	3.33	1584000	1.9278
2.0742	3.35	1592000	1.9333
2.0774	3.37	1600000	1.9252
2.0774	3.39	1608000	1.9301
2.0766	3.4	1616000	1.9344
2.0766	3.42	1624000	1.9320
2.0702	3.44	1632000	1.9307
2.0702	3.45	1640000	1.9304
2.0772	3.47	1648000	1.9280
2.0772	3.49	1656000	1.9324
2.0757	3.5	1664000	1.9343
2.0757	3.52	1672000	1.9312
2.0747	3.54	1680000	1.9304
2.0747	3.55	1688000	1.9360
2.068	3.57	1696000	1.9297
2.068	3.59	1704000	1.9337
2.0825	3.6	1712000	1.9293
2.0825	3.62	1720000	1.9295
2.0811	3.64	1728000	1.9315
2.0811	3.65	1736000	1.9279
2.0844	3.67	1744000	1.9289
2.0844	3.69	1752000	1.9279
2.0827	3.71	1760000	1.9283
2.0827	3.72	1768000	1.9295
2.0684	3.74	1776000	1.9281
2.0684	3.76	1784000	1.9330
2.0724	3.77	1792000	1.9294
2.0724	3.79	1800000	1.9276
2.074	3.81	1808000	1.9227
2.074	3.82	1816000	1.9320
2.0801	3.84	1824000	1.9275
2.0801	3.86	1832000	1.9302
2.0783	3.87	1840000	1.9333
2.0783	3.89	1848000	1.9296
2.0787	3.91	1856000	1.9302
2.0787	3.92	1864000	1.9347
2.0733	3.94	1872000	1.9298
2.0733	3.96	1880000	1.9302
2.0742	3.97	1888000	1.9279
2.0742	3.99	1896000	1.9258
2.0769	4.01	1904000	1.9255
2.0769	4.03	1912000	1.9282
2.0736	4.04	1920000	1.9298
2.0736	4.06	1928000	1.9325
2.0713	4.08	1936000	1.9296
2.0713	4.09	1944000	1.9293
2.0825	4.11	1952000	1.9345
2.0825	4.13	1960000	1.9346
2.0828	4.14	1968000	1.9311
2.0828	4.16	1976000	1.9307
2.0821	4.18	1984000	1.9336
2.0821	4.19	1992000	1.9265
2.0768	4.21	2000000	1.9284
2.0768	4.23	2008000	1.9290
2.0695	4.24	2016000	1.9306
2.0695	4.26	2024000	1.9299
2.0698	4.28	2032000	1.9230
2.0698	4.29	2040000	1.9272
2.0776	4.31	2048000	1.9306
2.0776	4.33	2056000	1.9243
2.0797	4.35	2064000	1.9266
2.0797	4.36	2072000	1.9249
2.0808	4.38	2080000	1.9279
2.0808	4.4	2088000	1.9262
2.0776	4.41	2096000	1.9350
2.0776	4.43	2104000	1.9297
2.0805	4.45	2112000	1.9337
2.0805	4.46	2120000	1.9302
2.0791	4.48	2128000	1.9337
2.0791	4.5	2136000	1.9298
2.0771	4.51	2144000	1.9268
2.0771	4.53	2152000	1.9370
2.0807	4.55	2160000	1.9307
2.0807	4.56	2168000	1.9292
2.0856	4.58	2176000	1.9300
2.0856	4.6	2184000	1.9329
2.0744	4.61	2192000	1.9319
2.0744	4.63	2200000	1.9352
2.0839	4.65	2208000	1.9368
2.0839	4.67	2216000	1.9343
2.0706	4.68	2224000	1.9290
2.0706	4.7	2232000	1.9347
2.0745	4.72	2240000	1.9294
2.0745	4.73	2248000	1.9255
2.0767	4.75	2256000	1.9271
2.0767	4.77	2264000	1.9296
2.0753	4.78	2272000	1.9268
2.0753	4.8	2280000	1.9292
2.0716	4.82	2288000	1.9310
2.0716	4.83	2296000	1.9267
2.0778	4.85	2304000	1.9301
2.0778	4.87	2312000	1.9280
2.0724	4.88	2320000	1.9283
2.0724	4.9	2328000	1.9289
2.0811	4.92	2336000	1.9315
2.0811	4.93	2344000	1.9268
2.0816	4.95	2352000	1.9304
2.0816	4.97	2360000	1.9302
2.0775	4.99	2368000	1.9292
2.0775	5.0	2376000	1.9274
2.0807	5.02	2384000	1.9317
2.0807	5.04	2392000	1.9298
2.0668	5.05	2400000	1.9349

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0