DouglasPontes/2020-Q1-50p-filtered-random

2020-Q1-50p-filtered

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.4514

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.03	8000	2.8937
3.073	0.07	16000	2.7660
3.073	0.1	24000	2.7233
2.8244	0.13	32000	2.6878
2.8244	0.16	40000	2.6520
2.7542	0.2	48000	2.6300
2.7542	0.23	56000	2.6135
2.7083	0.26	64000	2.6068
2.7083	0.3	72000	2.5854
2.6752	0.33	80000	2.5755
2.6752	0.36	88000	2.5721
2.6657	0.39	96000	2.5709
2.6657	0.43	104000	2.5656
2.6534	0.46	112000	2.5558
2.6534	0.49	120000	2.5496
2.646	0.52	128000	2.5471
2.646	0.56	136000	2.5408
2.625	0.59	144000	2.5315
2.625	0.62	152000	2.5365
2.6222	0.66	160000	2.5372
2.6222	0.69	168000	2.5342
2.6256	0.72	176000	2.5308
2.6256	0.75	184000	2.5312
2.6074	0.79	192000	2.5228
2.6074	0.82	200000	2.5292
2.6071	0.85	208000	2.5295
2.6071	0.89	216000	2.5235
2.5955	0.92	224000	2.5219
2.5955	0.95	232000	2.5191
2.6036	0.98	240000	2.5171
2.6036	1.02	248000	2.5102
2.6046	1.05	256000	2.5070
2.6046	1.08	264000	2.5109
2.5892	1.11	272000	2.5105
2.5892	1.15	280000	2.5087
2.5929	1.18	288000	2.5094
2.5929	1.21	296000	2.5086
2.5857	1.25	304000	2.4991
2.5857	1.28	312000	2.5089
2.5828	1.31	320000	2.5017
2.5828	1.34	328000	2.5039
2.5812	1.38	336000	2.5065
2.5812	1.41	344000	2.5083
2.5775	1.44	352000	2.5099
2.5775	1.48	360000	2.5079
2.5711	1.51	368000	2.4922
2.5711	1.54	376000	2.5012
2.5797	1.57	384000	2.4999
2.5797	1.61	392000	2.4881
2.5718	1.64	400000	2.4960
2.5718	1.67	408000	2.4908
2.5627	1.7	416000	2.4971
2.5627	1.74	424000	2.4916
2.5641	1.77	432000	2.4971
2.5641	1.8	440000	2.4954
2.5633	1.84	448000	2.4860
2.5633	1.87	456000	2.4894
2.5676	1.9	464000	2.4893
2.5676	1.93	472000	2.4884
2.5687	1.97	480000	2.4921
2.5687	2.0	488000	2.4873
2.5633	2.03	496000	2.4919
2.5633	2.07	504000	2.4821
2.5547	2.1	512000	2.4909
2.5547	2.13	520000	2.4818
2.5617	2.16	528000	2.4855
2.5617	2.2	536000	2.4850
2.5569	2.23	544000	2.4803
2.5569	2.26	552000	2.4776
2.5535	2.29	560000	2.4824
2.5535	2.33	568000	2.4822
2.5534	2.36	576000	2.4763
2.5534	2.39	584000	2.4797
2.5583	2.43	592000	2.4872
2.5583	2.46	600000	2.4812
2.5545	2.49	608000	2.4748
2.5545	2.52	616000	2.4736
2.5561	2.56	624000	2.4714
2.5561	2.59	632000	2.4858
2.5384	2.62	640000	2.4829
2.5384	2.66	648000	2.4766
2.541	2.69	656000	2.4836
2.541	2.72	664000	2.4651
2.5439	2.75	672000	2.4797
2.5439	2.79	680000	2.4702
2.5597	2.82	688000	2.4751
2.5597	2.85	696000	2.4744
2.5491	2.88	704000	2.4756
2.5491	2.92	712000	2.4731
2.5505	2.95	720000	2.4756
2.5505	2.98	728000	2.4704
2.5432	3.02	736000	2.4763
2.5432	3.05	744000	2.4743
2.5485	3.08	752000	2.4627
2.5485	3.11	760000	2.4714
2.5482	3.15	768000	2.4685
2.5482	3.18	776000	2.4673
2.5411	3.21	784000	2.4726
2.5411	3.25	792000	2.4761
2.5407	3.28	800000	2.4612
2.5407	3.31	808000	2.4743
2.5307	3.34	816000	2.4699
2.5307	3.38	824000	2.4721
2.5391	3.41	832000	2.4614
2.5391	3.44	840000	2.4641
2.5378	3.47	848000	2.4652
2.5378	3.51	856000	2.4641
2.5399	3.54	864000	2.4691
2.5399	3.57	872000	2.4612
2.5412	3.61	880000	2.4696
2.5412	3.64	888000	2.4638
2.5389	3.67	896000	2.4658
2.5389	3.7	904000	2.4725
2.5325	3.74	912000	2.4642
2.5325	3.77	920000	2.4599
2.5351	3.8	928000	2.4617
2.5351	3.84	936000	2.4646
2.522	3.87	944000	2.4665
2.522	3.9	952000	2.4762
2.5331	3.93	960000	2.4669
2.5331	3.97	968000	2.4550
2.5276	4.0	976000	2.4662
2.5276	4.03	984000	2.4645
2.5206	4.06	992000	2.4587
2.5206	4.1	1000000	2.4725
2.5294	4.13	1008000	2.4588
2.5294	4.16	1016000	2.4591
2.5312	4.2	1024000	2.4681
2.5312	4.23	1032000	2.4625
2.525	4.26	1040000	2.4659
2.525	4.29	1048000	2.4609
2.5318	4.33	1056000	2.4571
2.5318	4.36	1064000	2.4582
2.5332	4.39	1072000	2.4566
2.5332	4.43	1080000	2.4588
2.5168	4.46	1088000	2.4606
2.5168	4.49	1096000	2.4598
2.5181	4.52	1104000	2.4543
2.5181	4.56	1112000	2.4620
2.5246	4.59	1120000	2.4639
2.5246	4.62	1128000	2.4556
2.5318	4.65	1136000	2.4571
2.5318	4.69	1144000	2.4636
2.512	4.72	1152000	2.4568
2.512	4.75	1160000	2.4644
2.5174	4.79	1168000	2.4529
2.5174	4.82	1176000	2.4614
2.5196	4.85	1184000	2.4638
2.5196	4.88	1192000	2.4534
2.5248	4.92	1200000	2.4553
2.5248	4.95	1208000	2.4537
2.5201	4.98	1216000	2.4579
2.5201	5.02	1224000	2.4525
2.5164	5.05	1232000	2.4645
2.5164	5.08	1240000	2.4480
2.5186	5.11	1248000	2.4606
2.5186	5.15	1256000	2.4623
2.5123	5.18	1264000	2.4566
2.5123	5.21	1272000	2.4644
2.5233	5.24	1280000	2.4576
2.5233	5.28	1288000	2.4519
2.513	5.31	1296000	2.4570
2.513	5.34	1304000	2.4627
2.5226	5.38	1312000	2.4500
2.5226	5.41	1320000	2.4563
2.5222	5.44	1328000	2.4521
2.5222	5.47	1336000	2.4591
2.5191	5.51	1344000	2.4509
2.5191	5.54	1352000	2.4559
2.5243	5.57	1360000	2.4502
2.5243	5.61	1368000	2.4515
2.5157	5.64	1376000	2.4563
2.5157	5.67	1384000	2.4526
2.5162	5.7	1392000	2.4586
2.5162	5.74	1400000	2.4584
2.5169	5.77	1408000	2.4542
2.5169	5.8	1416000	2.4602
2.5127	5.84	1424000	2.4587
2.5127	5.87	1432000	2.4529
2.5144	5.9	1440000	2.4620
2.5144	5.93	1448000	2.4509
2.5175	5.97	1456000	2.4503
2.5175	6.0	1464000	2.4545
2.5147	6.03	1472000	2.4440
2.5147	6.06	1480000	2.4577
2.5128	6.1	1488000	2.4566
2.5128	6.13	1496000	2.4499
2.5168	6.16	1504000	2.4480
2.5168	6.2	1512000	2.4436
2.5225	6.23	1520000	2.4467
2.5225	6.26	1528000	2.4520
2.5135	6.29	1536000	2.4535
2.5135	6.33	1544000	2.4463
2.5161	6.36	1552000	2.4556
2.5161	6.39	1560000	2.4605
2.5144	6.43	1568000	2.4516
2.5144	6.46	1576000	2.4488
2.5209	6.49	1584000	2.4525
2.5209	6.52	1592000	2.4502
2.5102	6.56	1600000	2.4538
2.5102	6.59	1608000	2.4491
2.5176	6.62	1616000	2.4528
2.5176	6.65	1624000	2.4460
2.5208	6.69	1632000	2.4485
2.5208	6.72	1640000	2.4513
2.5064	6.75	1648000	2.4519
2.5064	6.79	1656000	2.4493
2.5111	6.82	1664000	2.4505
2.5111	6.85	1672000	2.4502
2.5141	6.88	1680000	2.4560
2.5141	6.92	1688000	2.4500
2.5089	6.95	1696000	2.4513
2.5089	6.98	1704000	2.4418
2.5174	7.02	1712000	2.4477
2.5174	7.05	1720000	2.4508
2.5198	7.08	1728000	2.4486
2.5198	7.11	1736000	2.4577
2.4974	7.15	1744000	2.4416
2.4974	7.18	1752000	2.4549
2.5016	7.21	1760000	2.4557
2.5016	7.24	1768000	2.4532
2.5112	7.28	1776000	2.4451
2.5112	7.31	1784000	2.4607
2.5172	7.34	1792000	2.4452
2.5172	7.38	1800000	2.4427
2.5089	7.41	1808000	2.4511
2.5089	7.44	1816000	2.4441
2.5136	7.47	1824000	2.4492
2.5136	7.51	1832000	2.4524
2.509	7.54	1840000	2.4512
2.509	7.57	1848000	2.4528
2.5157	7.61	1856000	2.4440
2.5157	7.64	1864000	2.4402
2.5181	7.67	1872000	2.4538
2.5181	7.7	1880000	2.4481
2.5145	7.74	1888000	2.4417
2.5145	7.77	1896000	2.4512
2.5013	7.8	1904000	2.4560
2.5013	7.83	1912000	2.4509
2.5064	7.87	1920000	2.4473
2.5064	7.9	1928000	2.4576
2.5068	7.93	1936000	2.4461
2.5068	7.97	1944000	2.4451
2.5152	8.0	1952000	2.4421
2.5152	8.03	1960000	2.4458
2.5025	8.06	1968000	2.4532
2.5025	8.1	1976000	2.4541
2.5151	8.13	1984000	2.4499
2.5151	8.16	1992000	2.4501
2.5138	8.2	2000000	2.4448
2.5138	8.23	2008000	2.4562
2.5039	8.26	2016000	2.4613
2.5039	8.29	2024000	2.4471
2.5055	8.33	2032000	2.4450
2.5055	8.36	2040000	2.4493
2.5085	8.39	2048000	2.4482
2.5085	8.42	2056000	2.4572
2.5114	8.46	2064000	2.4443
2.5114	8.49	2072000	2.4456
2.5132	8.52	2080000	2.4528
2.5132	8.56	2088000	2.4497
2.5072	8.59	2096000	2.4548
2.5072	8.62	2104000	2.4548
2.504	8.65	2112000	2.4443
2.504	8.69	2120000	2.4452
2.5128	8.72	2128000	2.4510
2.5128	8.75	2136000	2.4480
2.5133	8.79	2144000	2.4470
2.5133	8.82	2152000	2.4437
2.5067	8.85	2160000	2.4447
2.5067	8.88	2168000	2.4531
2.4996	8.92	2176000	2.4475
2.4996	8.95	2184000	2.4438
2.5123	8.98	2192000	2.4552
2.5123	9.01	2200000	2.4441
2.5044	9.05	2208000	2.4438
2.5044	9.08	2216000	2.4534
2.5068	9.11	2224000	2.4497
2.5068	9.15	2232000	2.4440
2.5165	9.18	2240000	2.4577
2.5165	9.21	2248000	2.4507
2.5087	9.24	2256000	2.4494
2.5087	9.28	2264000	2.4393
2.5036	9.31	2272000	2.4487
2.5036	9.34	2280000	2.4423
2.5086	9.38	2288000	2.4456
2.5086	9.41	2296000	2.4496
2.5034	9.44	2304000	2.4499
2.5034	9.47	2312000	2.4433
2.5099	9.51	2320000	2.4534
2.5099	9.54	2328000	2.4495
2.5065	9.57	2336000	2.4510
2.5065	9.6	2344000	2.4513
2.502	9.64	2352000	2.4512
2.502	9.67	2360000	2.4469
2.5043	9.7	2368000	2.4544
2.5043	9.74	2376000	2.4493
2.5068	9.77	2384000	2.4537
2.5068	9.8	2392000	2.4387
2.5118	9.83	2400000	2.4494

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0