--- license: mit base_model: cardiffnlp/twitter-roberta-base-2019-90m tags: - generated_from_trainer model-index: - name: 2020-Q1-50p-filtered results: [] --- # 2020-Q1-50p-filtered This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-2019-90m](https://huggingface.co/cardiffnlp/twitter-roberta-base-2019-90m) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 2.4514 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 4.1e-07 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08 - lr_scheduler_type: linear - training_steps: 2400000 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:-------:|:---------------:| | No log | 0.03 | 8000 | 2.8937 | | 3.073 | 0.07 | 16000 | 2.7660 | | 3.073 | 0.1 | 24000 | 2.7233 | | 2.8244 | 0.13 | 32000 | 2.6878 | | 2.8244 | 0.16 | 40000 | 2.6520 | | 2.7542 | 0.2 | 48000 | 2.6300 | | 2.7542 | 0.23 | 56000 | 2.6135 | | 2.7083 | 0.26 | 64000 | 2.6068 | | 2.7083 | 0.3 | 72000 | 2.5854 | | 2.6752 | 0.33 | 80000 | 2.5755 | | 2.6752 | 0.36 | 88000 | 2.5721 | | 2.6657 | 0.39 | 96000 | 2.5709 | | 2.6657 | 0.43 | 104000 | 2.5656 | | 2.6534 | 0.46 | 112000 | 2.5558 | | 2.6534 | 0.49 | 120000 | 2.5496 | | 2.646 | 0.52 | 128000 | 2.5471 | | 2.646 | 0.56 | 136000 | 2.5408 | | 2.625 | 0.59 | 144000 | 2.5315 | | 2.625 | 0.62 | 152000 | 2.5365 | | 2.6222 | 0.66 | 160000 | 2.5372 | | 2.6222 | 0.69 | 168000 | 2.5342 | | 2.6256 | 0.72 | 176000 | 2.5308 | | 2.6256 | 0.75 | 184000 | 2.5312 | | 2.6074 | 0.79 | 192000 | 2.5228 | | 2.6074 | 0.82 | 200000 | 2.5292 | | 2.6071 | 0.85 | 208000 | 2.5295 | | 2.6071 | 0.89 | 216000 | 2.5235 | | 2.5955 | 0.92 | 224000 | 2.5219 | | 2.5955 | 0.95 | 232000 | 2.5191 | | 2.6036 | 0.98 | 240000 | 2.5171 | | 2.6036 | 1.02 | 248000 | 2.5102 | | 2.6046 | 1.05 | 256000 | 2.5070 | | 2.6046 | 1.08 | 264000 | 2.5109 | | 2.5892 | 1.11 | 272000 | 2.5105 | | 2.5892 | 1.15 | 280000 | 2.5087 | | 2.5929 | 1.18 | 288000 | 2.5094 | | 2.5929 | 1.21 | 296000 | 2.5086 | | 2.5857 | 1.25 | 304000 | 2.4991 | | 2.5857 | 1.28 | 312000 | 2.5089 | | 2.5828 | 1.31 | 320000 | 2.5017 | | 2.5828 | 1.34 | 328000 | 2.5039 | | 2.5812 | 1.38 | 336000 | 2.5065 | | 2.5812 | 1.41 | 344000 | 2.5083 | | 2.5775 | 1.44 | 352000 | 2.5099 | | 2.5775 | 1.48 | 360000 | 2.5079 | | 2.5711 | 1.51 | 368000 | 2.4922 | | 2.5711 | 1.54 | 376000 | 2.5012 | | 2.5797 | 1.57 | 384000 | 2.4999 | | 2.5797 | 1.61 | 392000 | 2.4881 | | 2.5718 | 1.64 | 400000 | 2.4960 | | 2.5718 | 1.67 | 408000 | 2.4908 | | 2.5627 | 1.7 | 416000 | 2.4971 | | 2.5627 | 1.74 | 424000 | 2.4916 | | 2.5641 | 1.77 | 432000 | 2.4971 | | 2.5641 | 1.8 | 440000 | 2.4954 | | 2.5633 | 1.84 | 448000 | 2.4860 | | 2.5633 | 1.87 | 456000 | 2.4894 | | 2.5676 | 1.9 | 464000 | 2.4893 | | 2.5676 | 1.93 | 472000 | 2.4884 | | 2.5687 | 1.97 | 480000 | 2.4921 | | 2.5687 | 2.0 | 488000 | 2.4873 | | 2.5633 | 2.03 | 496000 | 2.4919 | | 2.5633 | 2.07 | 504000 | 2.4821 | | 2.5547 | 2.1 | 512000 | 2.4909 | | 2.5547 | 2.13 | 520000 | 2.4818 | | 2.5617 | 2.16 | 528000 | 2.4855 | | 2.5617 | 2.2 | 536000 | 2.4850 | | 2.5569 | 2.23 | 544000 | 2.4803 | | 2.5569 | 2.26 | 552000 | 2.4776 | | 2.5535 | 2.29 | 560000 | 2.4824 | | 2.5535 | 2.33 | 568000 | 2.4822 | | 2.5534 | 2.36 | 576000 | 2.4763 | | 2.5534 | 2.39 | 584000 | 2.4797 | | 2.5583 | 2.43 | 592000 | 2.4872 | | 2.5583 | 2.46 | 600000 | 2.4812 | | 2.5545 | 2.49 | 608000 | 2.4748 | | 2.5545 | 2.52 | 616000 | 2.4736 | | 2.5561 | 2.56 | 624000 | 2.4714 | | 2.5561 | 2.59 | 632000 | 2.4858 | | 2.5384 | 2.62 | 640000 | 2.4829 | | 2.5384 | 2.66 | 648000 | 2.4766 | | 2.541 | 2.69 | 656000 | 2.4836 | | 2.541 | 2.72 | 664000 | 2.4651 | | 2.5439 | 2.75 | 672000 | 2.4797 | | 2.5439 | 2.79 | 680000 | 2.4702 | | 2.5597 | 2.82 | 688000 | 2.4751 | | 2.5597 | 2.85 | 696000 | 2.4744 | | 2.5491 | 2.88 | 704000 | 2.4756 | | 2.5491 | 2.92 | 712000 | 2.4731 | | 2.5505 | 2.95 | 720000 | 2.4756 | | 2.5505 | 2.98 | 728000 | 2.4704 | | 2.5432 | 3.02 | 736000 | 2.4763 | | 2.5432 | 3.05 | 744000 | 2.4743 | | 2.5485 | 3.08 | 752000 | 2.4627 | | 2.5485 | 3.11 | 760000 | 2.4714 | | 2.5482 | 3.15 | 768000 | 2.4685 | | 2.5482 | 3.18 | 776000 | 2.4673 | | 2.5411 | 3.21 | 784000 | 2.4726 | | 2.5411 | 3.25 | 792000 | 2.4761 | | 2.5407 | 3.28 | 800000 | 2.4612 | | 2.5407 | 3.31 | 808000 | 2.4743 | | 2.5307 | 3.34 | 816000 | 2.4699 | | 2.5307 | 3.38 | 824000 | 2.4721 | | 2.5391 | 3.41 | 832000 | 2.4614 | | 2.5391 | 3.44 | 840000 | 2.4641 | | 2.5378 | 3.47 | 848000 | 2.4652 | | 2.5378 | 3.51 | 856000 | 2.4641 | | 2.5399 | 3.54 | 864000 | 2.4691 | | 2.5399 | 3.57 | 872000 | 2.4612 | | 2.5412 | 3.61 | 880000 | 2.4696 | | 2.5412 | 3.64 | 888000 | 2.4638 | | 2.5389 | 3.67 | 896000 | 2.4658 | | 2.5389 | 3.7 | 904000 | 2.4725 | | 2.5325 | 3.74 | 912000 | 2.4642 | | 2.5325 | 3.77 | 920000 | 2.4599 | | 2.5351 | 3.8 | 928000 | 2.4617 | | 2.5351 | 3.84 | 936000 | 2.4646 | | 2.522 | 3.87 | 944000 | 2.4665 | | 2.522 | 3.9 | 952000 | 2.4762 | | 2.5331 | 3.93 | 960000 | 2.4669 | | 2.5331 | 3.97 | 968000 | 2.4550 | | 2.5276 | 4.0 | 976000 | 2.4662 | | 2.5276 | 4.03 | 984000 | 2.4645 | | 2.5206 | 4.06 | 992000 | 2.4587 | | 2.5206 | 4.1 | 1000000 | 2.4725 | | 2.5294 | 4.13 | 1008000 | 2.4588 | | 2.5294 | 4.16 | 1016000 | 2.4591 | | 2.5312 | 4.2 | 1024000 | 2.4681 | | 2.5312 | 4.23 | 1032000 | 2.4625 | | 2.525 | 4.26 | 1040000 | 2.4659 | | 2.525 | 4.29 | 1048000 | 2.4609 | | 2.5318 | 4.33 | 1056000 | 2.4571 | | 2.5318 | 4.36 | 1064000 | 2.4582 | | 2.5332 | 4.39 | 1072000 | 2.4566 | | 2.5332 | 4.43 | 1080000 | 2.4588 | | 2.5168 | 4.46 | 1088000 | 2.4606 | | 2.5168 | 4.49 | 1096000 | 2.4598 | | 2.5181 | 4.52 | 1104000 | 2.4543 | | 2.5181 | 4.56 | 1112000 | 2.4620 | | 2.5246 | 4.59 | 1120000 | 2.4639 | | 2.5246 | 4.62 | 1128000 | 2.4556 | | 2.5318 | 4.65 | 1136000 | 2.4571 | | 2.5318 | 4.69 | 1144000 | 2.4636 | | 2.512 | 4.72 | 1152000 | 2.4568 | | 2.512 | 4.75 | 1160000 | 2.4644 | | 2.5174 | 4.79 | 1168000 | 2.4529 | | 2.5174 | 4.82 | 1176000 | 2.4614 | | 2.5196 | 4.85 | 1184000 | 2.4638 | | 2.5196 | 4.88 | 1192000 | 2.4534 | | 2.5248 | 4.92 | 1200000 | 2.4553 | | 2.5248 | 4.95 | 1208000 | 2.4537 | | 2.5201 | 4.98 | 1216000 | 2.4579 | | 2.5201 | 5.02 | 1224000 | 2.4525 | | 2.5164 | 5.05 | 1232000 | 2.4645 | | 2.5164 | 5.08 | 1240000 | 2.4480 | | 2.5186 | 5.11 | 1248000 | 2.4606 | | 2.5186 | 5.15 | 1256000 | 2.4623 | | 2.5123 | 5.18 | 1264000 | 2.4566 | | 2.5123 | 5.21 | 1272000 | 2.4644 | | 2.5233 | 5.24 | 1280000 | 2.4576 | | 2.5233 | 5.28 | 1288000 | 2.4519 | | 2.513 | 5.31 | 1296000 | 2.4570 | | 2.513 | 5.34 | 1304000 | 2.4627 | | 2.5226 | 5.38 | 1312000 | 2.4500 | | 2.5226 | 5.41 | 1320000 | 2.4563 | | 2.5222 | 5.44 | 1328000 | 2.4521 | | 2.5222 | 5.47 | 1336000 | 2.4591 | | 2.5191 | 5.51 | 1344000 | 2.4509 | | 2.5191 | 5.54 | 1352000 | 2.4559 | | 2.5243 | 5.57 | 1360000 | 2.4502 | | 2.5243 | 5.61 | 1368000 | 2.4515 | | 2.5157 | 5.64 | 1376000 | 2.4563 | | 2.5157 | 5.67 | 1384000 | 2.4526 | | 2.5162 | 5.7 | 1392000 | 2.4586 | | 2.5162 | 5.74 | 1400000 | 2.4584 | | 2.5169 | 5.77 | 1408000 | 2.4542 | | 2.5169 | 5.8 | 1416000 | 2.4602 | | 2.5127 | 5.84 | 1424000 | 2.4587 | | 2.5127 | 5.87 | 1432000 | 2.4529 | | 2.5144 | 5.9 | 1440000 | 2.4620 | | 2.5144 | 5.93 | 1448000 | 2.4509 | | 2.5175 | 5.97 | 1456000 | 2.4503 | | 2.5175 | 6.0 | 1464000 | 2.4545 | | 2.5147 | 6.03 | 1472000 | 2.4440 | | 2.5147 | 6.06 | 1480000 | 2.4577 | | 2.5128 | 6.1 | 1488000 | 2.4566 | | 2.5128 | 6.13 | 1496000 | 2.4499 | | 2.5168 | 6.16 | 1504000 | 2.4480 | | 2.5168 | 6.2 | 1512000 | 2.4436 | | 2.5225 | 6.23 | 1520000 | 2.4467 | | 2.5225 | 6.26 | 1528000 | 2.4520 | | 2.5135 | 6.29 | 1536000 | 2.4535 | | 2.5135 | 6.33 | 1544000 | 2.4463 | | 2.5161 | 6.36 | 1552000 | 2.4556 | | 2.5161 | 6.39 | 1560000 | 2.4605 | | 2.5144 | 6.43 | 1568000 | 2.4516 | | 2.5144 | 6.46 | 1576000 | 2.4488 | | 2.5209 | 6.49 | 1584000 | 2.4525 | | 2.5209 | 6.52 | 1592000 | 2.4502 | | 2.5102 | 6.56 | 1600000 | 2.4538 | | 2.5102 | 6.59 | 1608000 | 2.4491 | | 2.5176 | 6.62 | 1616000 | 2.4528 | | 2.5176 | 6.65 | 1624000 | 2.4460 | | 2.5208 | 6.69 | 1632000 | 2.4485 | | 2.5208 | 6.72 | 1640000 | 2.4513 | | 2.5064 | 6.75 | 1648000 | 2.4519 | | 2.5064 | 6.79 | 1656000 | 2.4493 | | 2.5111 | 6.82 | 1664000 | 2.4505 | | 2.5111 | 6.85 | 1672000 | 2.4502 | | 2.5141 | 6.88 | 1680000 | 2.4560 | | 2.5141 | 6.92 | 1688000 | 2.4500 | | 2.5089 | 6.95 | 1696000 | 2.4513 | | 2.5089 | 6.98 | 1704000 | 2.4418 | | 2.5174 | 7.02 | 1712000 | 2.4477 | | 2.5174 | 7.05 | 1720000 | 2.4508 | | 2.5198 | 7.08 | 1728000 | 2.4486 | | 2.5198 | 7.11 | 1736000 | 2.4577 | | 2.4974 | 7.15 | 1744000 | 2.4416 | | 2.4974 | 7.18 | 1752000 | 2.4549 | | 2.5016 | 7.21 | 1760000 | 2.4557 | | 2.5016 | 7.24 | 1768000 | 2.4532 | | 2.5112 | 7.28 | 1776000 | 2.4451 | | 2.5112 | 7.31 | 1784000 | 2.4607 | | 2.5172 | 7.34 | 1792000 | 2.4452 | | 2.5172 | 7.38 | 1800000 | 2.4427 | | 2.5089 | 7.41 | 1808000 | 2.4511 | | 2.5089 | 7.44 | 1816000 | 2.4441 | | 2.5136 | 7.47 | 1824000 | 2.4492 | | 2.5136 | 7.51 | 1832000 | 2.4524 | | 2.509 | 7.54 | 1840000 | 2.4512 | | 2.509 | 7.57 | 1848000 | 2.4528 | | 2.5157 | 7.61 | 1856000 | 2.4440 | | 2.5157 | 7.64 | 1864000 | 2.4402 | | 2.5181 | 7.67 | 1872000 | 2.4538 | | 2.5181 | 7.7 | 1880000 | 2.4481 | | 2.5145 | 7.74 | 1888000 | 2.4417 | | 2.5145 | 7.77 | 1896000 | 2.4512 | | 2.5013 | 7.8 | 1904000 | 2.4560 | | 2.5013 | 7.83 | 1912000 | 2.4509 | | 2.5064 | 7.87 | 1920000 | 2.4473 | | 2.5064 | 7.9 | 1928000 | 2.4576 | | 2.5068 | 7.93 | 1936000 | 2.4461 | | 2.5068 | 7.97 | 1944000 | 2.4451 | | 2.5152 | 8.0 | 1952000 | 2.4421 | | 2.5152 | 8.03 | 1960000 | 2.4458 | | 2.5025 | 8.06 | 1968000 | 2.4532 | | 2.5025 | 8.1 | 1976000 | 2.4541 | | 2.5151 | 8.13 | 1984000 | 2.4499 | | 2.5151 | 8.16 | 1992000 | 2.4501 | | 2.5138 | 8.2 | 2000000 | 2.4448 | | 2.5138 | 8.23 | 2008000 | 2.4562 | | 2.5039 | 8.26 | 2016000 | 2.4613 | | 2.5039 | 8.29 | 2024000 | 2.4471 | | 2.5055 | 8.33 | 2032000 | 2.4450 | | 2.5055 | 8.36 | 2040000 | 2.4493 | | 2.5085 | 8.39 | 2048000 | 2.4482 | | 2.5085 | 8.42 | 2056000 | 2.4572 | | 2.5114 | 8.46 | 2064000 | 2.4443 | | 2.5114 | 8.49 | 2072000 | 2.4456 | | 2.5132 | 8.52 | 2080000 | 2.4528 | | 2.5132 | 8.56 | 2088000 | 2.4497 | | 2.5072 | 8.59 | 2096000 | 2.4548 | | 2.5072 | 8.62 | 2104000 | 2.4548 | | 2.504 | 8.65 | 2112000 | 2.4443 | | 2.504 | 8.69 | 2120000 | 2.4452 | | 2.5128 | 8.72 | 2128000 | 2.4510 | | 2.5128 | 8.75 | 2136000 | 2.4480 | | 2.5133 | 8.79 | 2144000 | 2.4470 | | 2.5133 | 8.82 | 2152000 | 2.4437 | | 2.5067 | 8.85 | 2160000 | 2.4447 | | 2.5067 | 8.88 | 2168000 | 2.4531 | | 2.4996 | 8.92 | 2176000 | 2.4475 | | 2.4996 | 8.95 | 2184000 | 2.4438 | | 2.5123 | 8.98 | 2192000 | 2.4552 | | 2.5123 | 9.01 | 2200000 | 2.4441 | | 2.5044 | 9.05 | 2208000 | 2.4438 | | 2.5044 | 9.08 | 2216000 | 2.4534 | | 2.5068 | 9.11 | 2224000 | 2.4497 | | 2.5068 | 9.15 | 2232000 | 2.4440 | | 2.5165 | 9.18 | 2240000 | 2.4577 | | 2.5165 | 9.21 | 2248000 | 2.4507 | | 2.5087 | 9.24 | 2256000 | 2.4494 | | 2.5087 | 9.28 | 2264000 | 2.4393 | | 2.5036 | 9.31 | 2272000 | 2.4487 | | 2.5036 | 9.34 | 2280000 | 2.4423 | | 2.5086 | 9.38 | 2288000 | 2.4456 | | 2.5086 | 9.41 | 2296000 | 2.4496 | | 2.5034 | 9.44 | 2304000 | 2.4499 | | 2.5034 | 9.47 | 2312000 | 2.4433 | | 2.5099 | 9.51 | 2320000 | 2.4534 | | 2.5099 | 9.54 | 2328000 | 2.4495 | | 2.5065 | 9.57 | 2336000 | 2.4510 | | 2.5065 | 9.6 | 2344000 | 2.4513 | | 2.502 | 9.64 | 2352000 | 2.4512 | | 2.502 | 9.67 | 2360000 | 2.4469 | | 2.5043 | 9.7 | 2368000 | 2.4544 | | 2.5043 | 9.74 | 2376000 | 2.4493 | | 2.5068 | 9.77 | 2384000 | 2.4537 | | 2.5068 | 9.8 | 2392000 | 2.4387 | | 2.5118 | 9.83 | 2400000 | 2.4494 | ### Framework versions - Transformers 4.35.0.dev0 - Pytorch 2.0.1+cu117 - Datasets 2.14.5 - Tokenizers 0.14.0