DouglasPontes/2020-Q3-50p-filtered-random

2020-Q3-50p-filtered

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.6316

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.03	8000	2.9380
3.1308	0.07	16000	2.8593
3.1308	0.1	24000	2.8063
2.9519	0.14	32000	2.7832
2.9519	0.17	40000	2.7521
2.889	0.2	48000	2.7322
2.889	0.24	56000	2.7259
2.8592	0.27	64000	2.7218
2.8592	0.3	72000	2.7106
2.8345	0.34	80000	2.7071
2.8345	0.37	88000	2.6903
2.8303	0.41	96000	2.7001
2.8303	0.44	104000	2.6929
2.824	0.47	112000	2.6907
2.824	0.51	120000	2.6852
2.8223	0.54	128000	2.6804
2.8223	0.57	136000	2.6727
2.8141	0.61	144000	2.6784
2.8141	0.64	152000	2.6775
2.8124	0.68	160000	2.6723
2.8124	0.71	168000	2.6683
2.8042	0.74	176000	2.6712
2.8042	0.78	184000	2.6661
2.8051	0.81	192000	2.6783
2.8051	0.85	200000	2.6683
2.798	0.88	208000	2.6656
2.798	0.91	216000	2.6659
2.8043	0.95	224000	2.6700
2.8043	0.98	232000	2.6680
2.8055	1.01	240000	2.6597
2.8055	1.05	248000	2.6597
2.8048	1.08	256000	2.6569
2.8048	1.12	264000	2.6502
2.806	1.15	272000	2.6593
2.806	1.18	280000	2.6597
2.8012	1.22	288000	2.6604
2.8012	1.25	296000	2.6545
2.8029	1.28	304000	2.6571
2.8029	1.32	312000	2.6534
2.7991	1.35	320000	2.6650
2.7991	1.39	328000	2.6680
2.7949	1.42	336000	2.6544
2.7949	1.45	344000	2.6460
2.7972	1.49	352000	2.6553
2.7972	1.52	360000	2.6428
2.7924	1.56	368000	2.6536
2.7924	1.59	376000	2.6550
2.805	1.62	384000	2.6524
2.805	1.66	392000	2.6524
2.7972	1.69	400000	2.6579
2.7972	1.72	408000	2.6500
2.8003	1.76	416000	2.6526
2.8003	1.79	424000	2.6444
2.8005	1.83	432000	2.6463
2.8005	1.86	440000	2.6549
2.7957	1.89	448000	2.6530
2.7957	1.93	456000	2.6504
2.7949	1.96	464000	2.6480
2.7949	1.99	472000	2.6497
2.7978	2.03	480000	2.6490
2.7978	2.06	488000	2.6505
2.8041	2.1	496000	2.6388
2.8041	2.13	504000	2.6460
2.7935	2.16	512000	2.6519
2.7935	2.2	520000	2.6494
2.7982	2.23	528000	2.6550
2.7982	2.27	536000	2.6460
2.7949	2.3	544000	2.6497
2.7949	2.33	552000	2.6478
2.7953	2.37	560000	2.6487
2.7953	2.4	568000	2.6400
2.7942	2.43	576000	2.6440
2.7942	2.47	584000	nan
2.803	2.5	592000	2.6455
2.803	2.54	600000	2.6401
2.7961	2.57	608000	2.6511
2.7961	2.6	616000	2.6401
2.7975	2.64	624000	2.6437
2.7975	2.67	632000	2.6432
2.7946	2.7	640000	2.6461
2.7946	2.74	648000	2.6491
2.7963	2.77	656000	2.6442
2.7963	2.81	664000	2.6416
2.7924	2.84	672000	2.6403
2.7924	2.87	680000	2.6466
2.8004	2.91	688000	2.6436
2.8004	2.94	696000	2.6447
2.8039	2.98	704000	2.6412
2.8039	3.01	712000	2.6400
2.7958	3.04	720000	2.6419
2.7958	3.08	728000	2.6413
2.7967	3.11	736000	nan
2.7967	3.14	744000	2.6399
2.7934	3.18	752000	2.6405
2.7934	3.21	760000	2.6387
2.7988	3.25	768000	2.6463
2.7988	3.28	776000	2.6308
2.793	3.31	784000	2.6343
2.793	3.35	792000	2.6358
2.797	3.38	800000	2.6397
2.797	3.41	808000	2.6341
2.7832	3.45	816000	2.6394
2.7832	3.48	824000	2.6341
2.792	3.52	832000	2.6424
2.792	3.55	840000	2.6380
2.7945	3.58	848000	2.6373
2.7945	3.62	856000	2.6366
2.7876	3.65	864000	2.6409
2.7876	3.69	872000	2.6382
2.7975	3.72	880000	2.6259
2.7975	3.75	888000	2.6443
2.7965	3.79	896000	2.6248
2.7965	3.82	904000	2.6395
2.7991	3.85	912000	2.6325
2.7991	3.89	920000	2.6354
2.7947	3.92	928000	2.6342
2.7947	3.96	936000	2.6290
2.7977	3.99	944000	2.6315
2.7977	4.02	952000	2.6347
2.8	4.06	960000	2.6318
2.8	4.09	968000	2.6328
2.7945	4.12	976000	2.6315
2.7945	4.16	984000	2.6297
2.7946	4.19	992000	2.6378
2.7946	4.23	1000000	2.6328
2.7962	4.26	1008000	2.6296
2.7962	4.29	1016000	2.6347
2.7932	4.33	1024000	2.6355
2.7932	4.36	1032000	2.6364
2.7992	4.4	1040000	2.6327
2.7992	4.43	1048000	2.6273
2.7922	4.46	1056000	2.6301
2.7922	4.5	1064000	2.6350
2.7939	4.53	1072000	2.6358
2.7939	4.56	1080000	nan
2.789	4.6	1088000	2.6288
2.789	4.63	1096000	2.6267
2.7965	4.67	1104000	2.6229
2.7965	4.7	1112000	2.6331
2.7963	4.73	1120000	2.6368
2.7963	4.77	1128000	2.6436
2.7993	4.8	1136000	2.6363
2.7993	4.83	1144000	2.6288
2.7952	4.87	1152000	2.6294
2.7952	4.9	1160000	2.6337
2.7972	4.94	1168000	2.6235
2.7972	4.97	1176000	2.6405
2.7988	5.0	1184000	2.6266
2.7988	5.04	1192000	2.6328
2.7901	5.07	1200000	2.6335
2.7901	5.11	1208000	2.6405
2.7975	5.14	1216000	2.6246
2.7975	5.17	1224000	2.6315
2.7974	5.21	1232000	2.6390
2.7974	5.24	1240000	2.6318
2.7909	5.27	1248000	2.6237
2.7909	5.31	1256000	2.6343
2.7899	5.34	1264000	2.6288
2.7899	5.38	1272000	2.6297
2.7937	5.41	1280000	2.6343
2.7937	5.44	1288000	2.6306
2.7916	5.48	1296000	2.6268
2.7916	5.51	1304000	2.6317
2.7874	5.54	1312000	2.6380
2.7874	5.58	1320000	2.6281
2.7967	5.61	1328000	2.6334
2.7967	5.65	1336000	2.6273
2.791	5.68	1344000	2.6339
2.791	5.71	1352000	2.6276
2.791	5.75	1360000	2.6247
2.791	5.78	1368000	2.6303
2.7909	5.82	1376000	2.6355
2.7909	5.85	1384000	2.6352
2.7833	5.88	1392000	2.6321
2.7833	5.92	1400000	2.6336
2.7944	5.95	1408000	2.6312
2.7944	5.98	1416000	2.6223
2.8001	6.02	1424000	2.6369
2.8001	6.05	1432000	2.6299
2.7954	6.09	1440000	2.6373
2.7954	6.12	1448000	2.6223
2.7914	6.15	1456000	2.6225
2.7914	6.19	1464000	2.6277
2.7896	6.22	1472000	2.6334
2.7896	6.26	1480000	2.6260
2.7925	6.29	1488000	2.6312
2.7925	6.32	1496000	2.6336
2.7976	6.36	1504000	2.6270
2.7976	6.39	1512000	2.6286
2.8025	6.42	1520000	2.6320
2.8025	6.46	1528000	2.6252
2.7953	6.49	1536000	2.6319
2.7953	6.53	1544000	2.6223
2.7994	6.56	1552000	2.6358
2.7994	6.59	1560000	2.6296
2.7966	6.63	1568000	2.6360
2.7966	6.66	1576000	2.6327
2.7883	6.69	1584000	2.6365
2.7883	6.73	1592000	2.6258
2.7963	6.76	1600000	2.6401
2.7963	6.8	1608000	2.6318
2.7923	6.83	1616000	2.6330
2.7923	6.86	1624000	2.6372
2.789	6.9	1632000	2.6363
2.789	6.93	1640000	2.6346
2.7883	6.97	1648000	2.6292
2.7883	7.0	1656000	2.6284
2.7965	7.03	1664000	2.6408
2.7965	7.07	1672000	2.6296
2.7963	7.1	1680000	2.6331
2.7963	7.13	1688000	2.6339
2.7911	7.17	1696000	2.6206
2.7911	7.2	1704000	2.6268
2.794	7.24	1712000	2.6278
2.794	7.27	1720000	2.6242
2.7893	7.3	1728000	2.6329
2.7893	7.34	1736000	2.6342
2.7935	7.37	1744000	2.6329
2.7935	7.4	1752000	2.6294
2.7936	7.44	1760000	2.6301
2.7936	7.47	1768000	2.6295
2.7922	7.51	1776000	2.6261
2.7922	7.54	1784000	2.6370
2.7911	7.57	1792000	2.6364
2.7911	7.61	1800000	2.6232
2.795	7.64	1808000	2.6201
2.795	7.68	1816000	2.6329
2.7898	7.71	1824000	2.6249
2.7898	7.74	1832000	2.6249
2.7931	7.78	1840000	2.6361
2.7931	7.81	1848000	nan
2.7919	7.84	1856000	2.6270
2.7919	7.88	1864000	2.6362
2.7833	7.91	1872000	2.6278
2.7833	7.95	1880000	2.6232
2.8067	7.98	1888000	2.6260
2.8067	8.01	1896000	2.6262
2.7953	8.05	1904000	2.6271
2.7953	8.08	1912000	2.6270
2.7953	8.11	1920000	2.6305
2.7953	8.15	1928000	2.6254
2.7881	8.18	1936000	2.6297
2.7881	8.22	1944000	2.6271
2.7928	8.25	1952000	2.6254
2.7928	8.28	1960000	2.6286
2.8003	8.32	1968000	2.6330
2.8003	8.35	1976000	2.6286
2.7935	8.39	1984000	2.6408
2.7935	8.42	1992000	2.6275
2.7925	8.45	2000000	2.6259
2.7925	8.49	2008000	2.6302
2.7924	8.52	2016000	2.6320
2.7924	8.55	2024000	2.6295
2.799	8.59	2032000	2.6259
2.799	8.62	2040000	2.6246
2.7983	8.66	2048000	2.6295
2.7983	8.69	2056000	2.6194
2.7901	8.72	2064000	2.6258
2.7901	8.76	2072000	2.6334
2.7956	8.79	2080000	2.6361
2.7956	8.82	2088000	2.6177
2.8008	8.86	2096000	2.6322
2.8008	8.89	2104000	2.6281
2.791	8.93	2112000	2.6249
2.791	8.96	2120000	2.6284
2.7933	8.99	2128000	2.6270
2.7933	9.03	2136000	2.6241
2.7825	9.06	2144000	2.6254
2.7825	9.1	2152000	2.6283
2.7854	9.13	2160000	2.6343
2.7854	9.16	2168000	2.6208
2.7949	9.2	2176000	2.6293
2.7949	9.23	2184000	2.6266
2.7938	9.26	2192000	2.6270
2.7938	9.3	2200000	2.6238
2.7905	9.33	2208000	2.6282
2.7905	9.37	2216000	2.6246
2.8004	9.4	2224000	2.6274
2.8004	9.43	2232000	2.6252
2.7921	9.47	2240000	2.6343
2.7921	9.5	2248000	2.6328
2.7964	9.53	2256000	2.6206
2.7964	9.57	2264000	2.6235
2.7954	9.6	2272000	2.6288
2.7954	9.64	2280000	2.6204
2.7902	9.67	2288000	2.6232
2.7902	9.7	2296000	2.6239
2.8046	9.74	2304000	2.6241
2.8046	9.77	2312000	2.6259
2.793	9.81	2320000	2.6275
2.793	9.84	2328000	2.6264
2.7893	9.87	2336000	2.6332
2.7893	9.91	2344000	2.6214
2.7898	9.94	2352000	2.6318
2.7898	9.97	2360000	2.6239
2.7906	10.01	2368000	2.6215
2.7906	10.04	2376000	2.6336
2.7942	10.08	2384000	2.6218
2.7942	10.11	2392000	2.6299
2.7997	10.14	2400000	2.6303

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0