2020-Q4-25p-filtered-random

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2681

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.02	8000	2.5802
2.8151	0.04	16000	2.4882
2.8151	0.07	24000	2.4292
2.5636	0.09	32000	2.3980
2.5636	0.11	40000	2.3799
2.4947	0.13	48000	2.3665
2.4947	0.16	56000	2.3455
2.473	0.18	64000	2.3419
2.473	0.2	72000	2.3307
2.4512	0.22	80000	2.3289
2.4512	0.25	88000	2.3250
2.4421	0.27	96000	2.3189
2.4421	0.29	104000	2.3200
2.4354	0.31	112000	2.3155
2.4354	0.34	120000	2.3138
2.4324	0.36	128000	2.3054
2.4324	0.38	136000	2.3028
2.4253	0.4	144000	2.3029
2.4253	0.43	152000	2.3006
2.4156	0.45	160000	2.3001
2.4156	0.47	168000	2.2980
2.4165	0.49	176000	2.2913
2.4165	0.52	184000	2.2974
2.4131	0.54	192000	2.2906
2.4131	0.56	200000	2.2908
2.407	0.58	208000	2.2895
2.407	0.61	216000	2.2865
2.4153	0.63	224000	2.2914
2.4153	0.65	232000	2.2806
2.4011	0.67	240000	2.2819
2.4011	0.7	248000	2.2854
2.4087	0.72	256000	2.2837
2.4087	0.74	264000	2.2866
2.4059	0.76	272000	2.2855
2.4059	0.79	280000	2.2868
2.4086	0.81	288000	2.2770
2.4086	0.83	296000	2.2789
2.4093	0.85	304000	2.2792
2.4093	0.88	312000	2.2797
2.4036	0.9	320000	2.2794
2.4036	0.92	328000	2.2768
2.4063	0.94	336000	2.2836
2.4063	0.97	344000	2.2809
2.4047	0.99	352000	2.2808
2.4047	1.01	360000	2.2840
2.4084	1.03	368000	2.2799
2.4084	1.06	376000	2.2726
2.4041	1.08	384000	2.2824
2.4041	1.1	392000	2.2781
2.4034	1.12	400000	2.2751
2.4034	1.15	408000	2.2761
2.3951	1.17	416000	2.2732
2.3951	1.19	424000	2.2710
2.409	1.21	432000	2.2780
2.409	1.24	440000	2.2715
2.3985	1.26	448000	2.2790
2.3985	1.28	456000	2.2766
2.4016	1.3	464000	2.2745
2.4016	1.32	472000	2.2719
2.3978	1.35	480000	2.2755
2.3978	1.37	488000	2.2699
2.406	1.39	496000	2.2823
2.406	1.41	504000	2.2736
2.3958	1.44	512000	2.2728
2.3958	1.46	520000	2.2763
2.406	1.48	528000	2.2781
2.406	1.5	536000	2.2723
2.4	1.53	544000	2.2733
2.4	1.55	552000	2.2715
2.3998	1.57	560000	2.2716
2.3998	1.59	568000	2.2751
2.4017	1.62	576000	2.2743
2.4017	1.64	584000	2.2739
2.4019	1.66	592000	2.2755
2.4019	1.68	600000	2.2691
2.398	1.71	608000	2.2706
2.398	1.73	616000	2.2703
2.4027	1.75	624000	2.2657
2.4027	1.77	632000	2.2674
2.4	1.8	640000	2.2749
2.4	1.82	648000	2.2714
2.4046	1.84	656000	2.2695
2.4046	1.86	664000	2.2724
2.4033	1.89	672000	2.2697
2.4033	1.91	680000	2.2697
2.3981	1.93	688000	2.2674
2.3981	1.95	696000	2.2669
2.4029	1.98	704000	2.2755
2.4029	2.0	712000	2.2664
2.4046	2.02	720000	2.2759
2.4046	2.04	728000	2.2689
2.4056	2.07	736000	2.2710
2.4056	2.09	744000	2.2744
2.4036	2.11	752000	2.2653
2.4036	2.13	760000	2.2642
2.3961	2.16	768000	2.2703
2.3961	2.18	776000	2.2683
2.3939	2.2	784000	2.2746
2.3939	2.22	792000	2.2667
2.3998	2.25	800000	2.2690
2.3998	2.27	808000	2.2697
2.3921	2.29	816000	2.2681
2.3921	2.31	824000	2.2740
2.4011	2.34	832000	2.2704
2.4011	2.36	840000	2.2666
2.3948	2.38	848000	2.2689
2.3948	2.4	856000	2.2742
2.3957	2.43	864000	2.2755
2.3957	2.45	872000	2.2689
2.3971	2.47	880000	2.2717
2.3971	2.49	888000	2.2690
2.3982	2.52	896000	2.2645
2.3982	2.54	904000	2.2726
2.4005	2.56	912000	2.2628
2.4005	2.58	920000	2.2726
2.4037	2.6	928000	2.2760
2.4037	2.63	936000	2.2662
2.4031	2.65	944000	2.2729
2.4031	2.67	952000	2.2706
2.4025	2.69	960000	2.2684
2.4025	2.72	968000	2.2635
2.409	2.74	976000	2.2606
2.409	2.76	984000	2.2664
2.4085	2.78	992000	2.2647
2.4085	2.81	1000000	2.2656
2.3971	2.83	1008000	2.2655
2.3971	2.85	1016000	2.2681
2.3946	2.87	1024000	2.2671
2.3946	2.9	1032000	2.2660
2.4063	2.92	1040000	2.2697
2.4063	2.94	1048000	2.2706
2.399	2.96	1056000	2.2625
2.399	2.99	1064000	2.2699
2.4024	3.01	1072000	2.2622
2.4024	3.03	1080000	2.2695
2.4035	3.05	1088000	2.2700
2.4035	3.08	1096000	2.2624
2.4061	3.1	1104000	2.2690
2.4061	3.12	1112000	2.2653
2.4044	3.14	1120000	2.2679
2.4044	3.17	1128000	2.2658
2.3996	3.19	1136000	2.2680
2.3996	3.21	1144000	2.2668
2.3943	3.23	1152000	2.2689
2.3943	3.26	1160000	2.2702
2.3948	3.28	1168000	2.2653
2.3948	3.3	1176000	2.2621
2.4047	3.32	1184000	2.2723
2.4047	3.35	1192000	2.2718
2.4057	3.37	1200000	2.2668
2.4057	3.39	1208000	2.2649
2.3901	3.41	1216000	2.2699
2.3901	3.44	1224000	2.2683
2.3942	3.46	1232000	2.2679
2.3942	3.48	1240000	2.2647
2.4052	3.5	1248000	2.2656
2.4052	3.53	1256000	2.2679
2.401	3.55	1264000	2.2685
2.401	3.57	1272000	2.2654
2.4012	3.59	1280000	2.2607
2.4012	3.62	1288000	2.2668
2.4015	3.64	1296000	2.2672
2.4015	3.66	1304000	2.2685
2.4039	3.68	1312000	2.2675
2.4039	3.71	1320000	2.2702
2.3927	3.73	1328000	2.2689
2.3927	3.75	1336000	2.2674
2.3998	3.77	1344000	2.2694
2.3998	3.8	1352000	2.2649
2.404	3.82	1360000	2.2635
2.404	3.84	1368000	2.2681
2.4023	3.86	1376000	2.2601
2.4023	3.88	1384000	2.2661
2.393	3.91	1392000	2.2613
2.393	3.93	1400000	2.2717
2.402	3.95	1408000	2.2672
2.402	3.97	1416000	2.2637
2.4047	4.0	1424000	2.2705
2.4047	4.02	1432000	2.2682
2.4045	4.04	1440000	2.2630
2.4045	4.06	1448000	2.2699
2.3973	4.09	1456000	2.2579
2.3973	4.11	1464000	2.2601
2.399	4.13	1472000	2.2609
2.399	4.15	1480000	2.2697
2.399	4.18	1488000	2.2630
2.399	4.2	1496000	2.2658
2.3995	4.22	1504000	2.2656
2.3995	4.24	1512000	2.2689
2.3929	4.27	1520000	2.2678
2.3929	4.29	1528000	2.2694
2.404	4.31	1536000	2.2632
2.404	4.33	1544000	2.2657
2.3932	4.36	1552000	2.2642
2.3932	4.38	1560000	2.2607
2.3985	4.4	1568000	2.2635
2.3985	4.42	1576000	2.2645
2.3997	4.45	1584000	2.2654
2.3997	4.47	1592000	2.2672
2.396	4.49	1600000	2.2666
2.396	4.51	1608000	2.2708
2.4012	4.54	1616000	2.2707
2.4012	4.56	1624000	2.2684
2.4074	4.58	1632000	2.2676
2.4074	4.6	1640000	2.2658
2.3965	4.63	1648000	2.2716
2.3965	4.65	1656000	2.2656
2.4021	4.67	1664000	2.2690
2.4021	4.69	1672000	2.2656
2.3981	4.72	1680000	2.2659
2.3981	4.74	1688000	2.2667
2.3974	4.76	1696000	2.2655
2.3974	4.78	1704000	2.2676
2.3964	4.81	1712000	2.2655
2.3964	4.83	1720000	2.2636
2.3933	4.85	1728000	2.2679
2.3933	4.87	1736000	2.2667
2.4066	4.9	1744000	2.2647
2.4066	4.92	1752000	2.2657
2.4027	4.94	1760000	2.2628
2.4027	4.96	1768000	2.2642
2.4029	4.99	1776000	2.2677
2.4029	5.01	1784000	2.2704
2.3958	5.03	1792000	2.2650
2.3958	5.05	1800000	2.2650
2.4054	5.08	1808000	2.2680
2.4054	5.1	1816000	2.2601
2.3984	5.12	1824000	2.2671
2.3984	5.14	1832000	2.2639
2.4005	5.16	1840000	2.2629
2.4005	5.19	1848000	2.2656
2.3962	5.21	1856000	2.2646
2.3962	5.23	1864000	2.2571
2.4033	5.25	1872000	2.2689
2.4033	5.28	1880000	2.2632
2.4064	5.3	1888000	2.2633
2.4064	5.32	1896000	2.2694
2.3967	5.34	1904000	2.2685
2.3967	5.37	1912000	2.2636
2.4002	5.39	1920000	2.2687
2.4002	5.41	1928000	2.2632
2.4045	5.43	1936000	2.2625
2.4045	5.46	1944000	2.2677
2.4096	5.48	1952000	2.2563
2.4096	5.5	1960000	2.2642
2.4004	5.52	1968000	2.2692
2.4004	5.55	1976000	2.2696
2.4065	5.57	1984000	2.2579
2.4065	5.59	1992000	2.2660
2.4025	5.61	2000000	2.2654
2.4025	5.64	2008000	2.2706
2.3993	5.66	2016000	2.2704
2.3993	5.68	2024000	2.2664
2.4034	5.7	2032000	2.2659
2.4034	5.73	2040000	2.2680
2.4004	5.75	2048000	2.2611
2.4004	5.77	2056000	2.2646
2.4025	5.79	2064000	2.2682
2.4025	5.82	2072000	2.2646
2.4063	5.84	2080000	2.2598
2.4063	5.86	2088000	2.2673
2.4071	5.88	2096000	2.2646
2.4071	5.91	2104000	2.2672
2.401	5.93	2112000	2.2648
2.401	5.95	2120000	2.2654
2.402	5.97	2128000	2.2664
2.402	6.0	2136000	2.2683
2.4004	6.02	2144000	2.2618
2.4004	6.04	2152000	2.2669
2.4001	6.06	2160000	2.2630
2.4001	6.09	2168000	2.2632
2.4046	6.11	2176000	2.2696
2.4046	6.13	2184000	2.2641
2.405	6.15	2192000	2.2627
2.405	6.18	2200000	2.2681
2.4063	6.2	2208000	2.2604
2.4063	6.22	2216000	2.2715
2.3991	6.24	2224000	2.2683
2.3991	6.27	2232000	2.2657
2.405	6.29	2240000	2.2645
2.405	6.31	2248000	2.2676
2.3941	6.33	2256000	2.2706
2.3941	6.36	2264000	2.2593
2.4041	6.38	2272000	2.2679
2.4041	6.4	2280000	2.2643
2.4001	6.42	2288000	2.2728
2.4001	6.44	2296000	2.2631
2.3983	6.47	2304000	2.2636
2.3983	6.49	2312000	2.2630
2.4003	6.51	2320000	2.2663
2.4003	6.53	2328000	2.2647
2.3981	6.56	2336000	2.2669
2.3981	6.58	2344000	2.2660
2.3951	6.6	2352000	2.2692
2.3951	6.62	2360000	2.2644
2.4013	6.65	2368000	2.2610
2.4013	6.67	2376000	2.2655
2.4	6.69	2384000	2.2592
2.4	6.71	2392000	2.2666
2.3975	6.74	2400000	2.2685

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0