2020-Q3-75p-filtered-random

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2615

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.07	8000	2.5628
2.7869	0.13	16000	2.4609
2.7869	0.2	24000	2.4087
2.5402	0.27	32000	2.3783
2.5402	0.34	40000	2.3566
2.4735	0.4	48000	2.3468
2.4735	0.47	56000	2.3302
2.4558	0.54	64000	2.3247
2.4558	0.61	72000	2.3147
2.428	0.67	80000	2.3084
2.428	0.74	88000	2.3121
2.4192	0.81	96000	2.2980
2.4192	0.88	104000	2.3001
2.4125	0.94	112000	2.2931
2.4125	1.01	120000	2.2866
2.405	1.08	128000	2.2877
2.405	1.15	136000	2.2802
2.4039	1.21	144000	2.2891
2.4039	1.28	152000	2.2845
2.4028	1.35	160000	2.2820
2.4028	1.41	168000	2.2772
2.3948	1.48	176000	2.2839
2.3948	1.55	184000	2.2781
2.3911	1.62	192000	2.2832
2.3911	1.68	200000	2.2807
2.3997	1.75	208000	2.2696
2.3997	1.82	216000	2.2761
2.396	1.89	224000	2.2802
2.396	1.95	232000	2.2751
2.3884	2.02	240000	2.2608
2.3884	2.09	248000	2.2731
2.3915	2.16	256000	2.2729
2.3915	2.22	264000	2.2635
2.3897	2.29	272000	2.2700
2.3897	2.36	280000	2.2724
2.3897	2.43	288000	2.2653
2.3897	2.49	296000	2.2728
2.3887	2.56	304000	2.2643
2.3887	2.63	312000	2.2771
2.3999	2.69	320000	2.2671
2.3999	2.76	328000	2.2627
2.3855	2.83	336000	2.2652
2.3855	2.9	344000	2.2742
2.3917	2.96	352000	2.2724
2.3917	3.03	360000	2.2707
2.3919	3.1	368000	2.2675
2.3919	3.17	376000	2.2689
2.385	3.23	384000	2.2607
2.385	3.3	392000	2.2593
2.3986	3.37	400000	2.2710
2.3986	3.44	408000	2.2671
2.3887	3.5	416000	2.2654
2.3887	3.57	424000	2.2670
2.3929	3.64	432000	2.2608
2.3929	3.71	440000	2.2571
2.3858	3.77	448000	2.2670
2.3858	3.84	456000	2.2712
2.3848	3.91	464000	2.2606
2.3848	3.97	472000	2.2573
2.3901	4.04	480000	2.2588
2.3901	4.11	488000	2.2723
2.3921	4.18	496000	2.2702
2.3921	4.24	504000	2.2603
2.3909	4.31	512000	2.2709
2.3909	4.38	520000	2.2622
2.3907	4.45	528000	2.2623
2.3907	4.51	536000	2.2527
2.3963	4.58	544000	2.2632
2.3963	4.65	552000	2.2564
2.3881	4.72	560000	2.2688
2.3881	4.78	568000	2.2603
2.3934	4.85	576000	2.2607
2.3934	4.92	584000	2.2736
2.396	4.99	592000	2.2664
2.396	5.05	600000	2.2617
2.3959	5.12	608000	2.2672
2.3959	5.19	616000	2.2532
2.3931	5.25	624000	2.2590
2.3931	5.32	632000	2.2646
2.3804	5.39	640000	2.2660
2.3804	5.46	648000	2.2693
2.4016	5.52	656000	2.2573
2.4016	5.59	664000	2.2756
2.3882	5.66	672000	2.2716
2.3882	5.73	680000	2.2666
2.3893	5.79	688000	2.2587
2.3893	5.86	696000	2.2611
2.3894	5.93	704000	2.2629
2.3894	6.0	712000	2.2570
2.3909	6.06	720000	2.2706
2.3909	6.13	728000	2.2581
2.3872	6.2	736000	2.2646
2.3872	6.27	744000	2.2650
2.3835	6.33	752000	2.2599
2.3835	6.4	760000	2.2652
2.402	6.47	768000	2.2659
2.402	6.53	776000	2.2677
2.3953	6.6	784000	2.2576
2.3953	6.67	792000	2.2542
2.3907	6.74	800000	2.2689
2.3907	6.8	808000	2.2679
2.396	6.87	816000	2.2625
2.396	6.94	824000	2.2674
2.3913	7.01	832000	2.2674
2.3913	7.07	840000	2.2633
2.3842	7.14	848000	2.2660
2.3842	7.21	856000	2.2523
2.3921	7.28	864000	2.2599
2.3921	7.34	872000	2.2637
2.3938	7.41	880000	2.2561
2.3938	7.48	888000	2.2626
2.3909	7.55	896000	2.2592
2.3909	7.61	904000	2.2584
2.3928	7.68	912000	2.2750
2.3928	7.75	920000	2.2537
2.3972	7.81	928000	2.2557
2.3972	7.88	936000	2.2580
2.3841	7.95	944000	2.2663
2.3841	8.02	952000	2.2629
2.3871	8.08	960000	2.2665
2.3871	8.15	968000	2.2649
2.39	8.22	976000	2.2571
2.39	8.29	984000	2.2716
2.3932	8.35	992000	2.2613
2.3932	8.42	1000000	2.2683
2.394	8.49	1008000	2.2625
2.394	8.56	1016000	2.2633
2.3961	8.62	1024000	2.2699
2.3961	8.69	1032000	2.2633
2.3907	8.76	1040000	2.2613
2.3907	8.83	1048000	2.2643
2.3909	8.89	1056000	2.2627
2.3909	8.96	1064000	2.2649
2.3901	9.03	1072000	2.2723
2.3901	9.09	1080000	2.2608
2.3967	9.16	1088000	2.2591
2.3967	9.23	1096000	2.2616
2.4001	9.3	1104000	2.2681
2.4001	9.36	1112000	2.2590
2.394	9.43	1120000	2.2648
2.394	9.5	1128000	2.2643
2.3873	9.57	1136000	2.2620
2.3873	9.63	1144000	2.2731
2.3935	9.7	1152000	2.2708
2.3935	9.77	1160000	2.2662
2.3966	9.84	1168000	2.2691
2.3966	9.9	1176000	2.2612
2.3917	9.97	1184000	2.2578
2.3917	10.04	1192000	2.2568
2.3963	10.11	1200000	2.2574
2.3963	10.17	1208000	2.2652
2.3939	10.24	1216000	2.2679
2.3939	10.31	1224000	2.2603
2.4007	10.37	1232000	2.2602
2.4007	10.44	1240000	2.2560
2.4015	10.51	1248000	2.2675
2.4015	10.58	1256000	2.2595
2.3907	10.64	1264000	2.2653
2.3907	10.71	1272000	2.2662
2.3952	10.78	1280000	2.2702
2.3952	10.85	1288000	2.2646
2.3992	10.91	1296000	2.2658
2.3992	10.98	1304000	2.2618
2.3915	11.05	1312000	2.2714
2.3915	11.12	1320000	2.2594
2.3929	11.18	1328000	2.2620
2.3929	11.25	1336000	2.2745
2.3947	11.32	1344000	2.2540
2.3947	11.39	1352000	2.2611
2.3947	11.45	1360000	2.2606
2.3947	11.52	1368000	2.2669
2.3936	11.59	1376000	2.2582
2.3936	11.65	1384000	2.2598
2.3971	11.72	1392000	2.2617
2.3971	11.79	1400000	2.2661
2.3957	11.86	1408000	2.2691
2.3957	11.92	1416000	2.2643
2.3944	11.99	1424000	2.2600
2.3944	12.06	1432000	2.2640
2.394	12.13	1440000	2.2629
2.394	12.19	1448000	2.2607
2.3928	12.26	1456000	2.2624
2.3928	12.33	1464000	2.2681
2.3937	12.4	1472000	2.2594
2.3937	12.46	1480000	2.2717
2.3957	12.53	1488000	2.2642
2.3957	12.6	1496000	2.2635
2.3911	12.67	1504000	2.2662
2.3911	12.73	1512000	2.2568
2.4001	12.8	1520000	2.2622
2.4001	12.87	1528000	2.2640
2.3923	12.93	1536000	2.2593
2.3923	13.0	1544000	2.2544
2.4007	13.07	1552000	2.2658
2.4007	13.14	1560000	2.2587
2.3995	13.2	1568000	2.2668
2.3995	13.27	1576000	2.2682
2.3935	13.34	1584000	2.2603
2.3935	13.41	1592000	2.2618
2.3964	13.47	1600000	2.2716
2.3964	13.54	1608000	2.2563
2.3917	13.61	1616000	2.2674
2.3917	13.68	1624000	2.2617
2.3926	13.74	1632000	2.2668
2.3926	13.81	1640000	2.2614
2.3948	13.88	1648000	2.2628
2.3948	13.95	1656000	2.2671
2.3936	14.01	1664000	2.2652
2.3936	14.08	1672000	2.2705
2.3908	14.15	1680000	2.2670
2.3908	14.21	1688000	2.2657
2.3923	14.28	1696000	2.2699
2.3923	14.35	1704000	2.2700
2.3981	14.42	1712000	2.2667
2.3981	14.48	1720000	2.2707
2.3956	14.55	1728000	2.2558
2.3956	14.62	1736000	2.2721
2.3943	14.69	1744000	2.2676
2.3943	14.75	1752000	2.2675
2.3999	14.82	1760000	2.2556
2.3999	14.89	1768000	2.2655
2.4005	14.96	1776000	2.2769
2.4005	15.02	1784000	2.2697
2.3997	15.09	1792000	2.2614
2.3997	15.16	1800000	2.2669
2.399	15.23	1808000	2.2655
2.399	15.29	1816000	2.2636
2.3987	15.36	1824000	2.2689
2.3987	15.43	1832000	2.2589
2.3984	15.49	1840000	2.2662
2.3984	15.56	1848000	2.2635
2.3939	15.63	1856000	2.2629
2.3939	15.7	1864000	2.2660
2.3954	15.76	1872000	2.2600
2.3954	15.83	1880000	2.2589
2.3987	15.9	1888000	2.2564
2.3987	15.97	1896000	2.2588
2.3992	16.03	1904000	2.2662
2.3992	16.1	1912000	2.2688
2.3962	16.17	1920000	2.2675
2.3962	16.24	1928000	2.2538
2.404	16.3	1936000	2.2639
2.404	16.37	1944000	2.2707
2.3959	16.44	1952000	2.2590
2.3959	16.51	1960000	2.2653
2.3966	16.57	1968000	2.2748
2.3966	16.64	1976000	2.2640
2.3972	16.71	1984000	2.2764
2.3972	16.77	1992000	2.2725
2.398	16.84	2000000	2.2563
2.398	16.91	2008000	2.2614
2.3948	16.98	2016000	2.2725
2.3948	17.04	2024000	2.2718
2.3894	17.11	2032000	2.2684
2.3894	17.18	2040000	2.2638
2.3958	17.25	2048000	2.2723
2.3958	17.31	2056000	2.2640
2.3974	17.38	2064000	2.2530
2.3974	17.45	2072000	2.2635
2.3955	17.52	2080000	2.2654
2.3955	17.58	2088000	2.2647
2.4052	17.65	2096000	2.2608
2.4052	17.72	2104000	2.2637
2.3956	17.79	2112000	2.2660
2.3956	17.85	2120000	2.2645
2.3906	17.92	2128000	2.2618
2.3906	17.99	2136000	2.2579
2.3958	18.05	2144000	2.2709
2.3958	18.12	2152000	2.2715
2.3904	18.19	2160000	2.2602
2.3904	18.26	2168000	2.2671
2.4007	18.32	2176000	2.2658
2.4007	18.39	2184000	2.2650
2.3948	18.46	2192000	2.2659
2.3948	18.53	2200000	2.2599
2.3948	18.59	2208000	2.2651
2.3948	18.66	2216000	2.2632
2.3973	18.73	2224000	2.2714
2.3973	18.8	2232000	2.2654
2.3975	18.86	2240000	2.2557
2.3975	18.93	2248000	2.2611
2.3971	19.0	2256000	2.2715
2.3971	19.07	2264000	2.2626
2.3929	19.13	2272000	2.2604
2.3929	19.2	2280000	2.2772
2.3971	19.27	2288000	2.2680
2.3971	19.33	2296000	2.2633
2.3934	19.4	2304000	2.2566
2.3934	19.47	2312000	2.2527
2.3997	19.54	2320000	2.2630
2.3997	19.6	2328000	2.2605
2.3918	19.67	2336000	2.2585
2.3918	19.74	2344000	2.2656
2.3986	19.81	2352000	2.2587
2.3986	19.87	2360000	2.2652
2.4006	19.94	2368000	2.2616
2.4006	20.01	2376000	2.2572
2.3963	20.08	2384000	2.2667
2.3963	20.14	2392000	2.2603
2.4022	20.21	2400000	2.2687

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0