2020-Q4-75p-filtered-random

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2679

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.07	8000	2.5786
2.8197	0.13	16000	2.4788
2.8197	0.2	24000	2.4343
2.5564	0.27	32000	2.4143
2.5564	0.34	40000	2.3826
2.4967	0.4	48000	2.3655
2.4967	0.47	56000	2.3450
2.476	0.54	64000	2.3501
2.476	0.61	72000	2.3315
2.4525	0.67	80000	2.3286
2.4525	0.74	88000	2.3278
2.445	0.81	96000	2.3187
2.445	0.88	104000	2.3113
2.438	0.94	112000	2.3129
2.438	1.01	120000	2.3161
2.4233	1.08	128000	2.3009
2.4233	1.15	136000	2.3072
2.4182	1.21	144000	2.3069
2.4182	1.28	152000	2.3060
2.418	1.35	160000	2.2963
2.418	1.41	168000	2.3017
2.4106	1.48	176000	2.2863
2.4106	1.55	184000	2.2871
2.4093	1.62	192000	2.2870
2.4093	1.68	200000	2.2845
2.4124	1.75	208000	2.2971
2.4124	1.82	216000	2.2833
2.4031	1.89	224000	2.2866
2.4031	1.95	232000	2.2833
2.4056	2.02	240000	2.2877
2.4056	2.09	248000	2.2789
2.4035	2.16	256000	2.2872
2.4035	2.22	264000	2.2771
2.4068	2.29	272000	2.2824
2.4068	2.36	280000	2.2681
2.4069	2.43	288000	2.2866
2.4069	2.49	296000	2.2838
2.4059	2.56	304000	2.2804
2.4059	2.63	312000	2.2757
2.3997	2.69	320000	2.2775
2.3997	2.76	328000	2.2693
2.4025	2.83	336000	2.2751
2.4025	2.9	344000	2.2686
2.399	2.96	352000	2.2784
2.399	3.03	360000	2.2782
2.3953	3.1	368000	2.2694
2.3953	3.17	376000	2.2638
2.4002	3.23	384000	2.2785
2.4002	3.3	392000	2.2785
2.4035	3.37	400000	2.2774
2.4035	3.44	408000	2.2736
2.3985	3.5	416000	2.2808
2.3985	3.57	424000	2.2672
2.3996	3.64	432000	2.2765
2.3996	3.71	440000	2.2748
2.4052	3.77	448000	2.2647
2.4052	3.84	456000	2.2776
2.4025	3.91	464000	2.2734
2.4025	3.97	472000	2.2588
2.4082	4.04	480000	2.2724
2.4082	4.11	488000	2.2740
2.3993	4.18	496000	2.2726
2.3993	4.24	504000	2.2827
2.4029	4.31	512000	2.2728
2.4029	4.38	520000	2.2833
2.407	4.45	528000	2.2636
2.407	4.51	536000	2.2689
2.4039	4.58	544000	2.2741
2.4039	4.65	552000	2.2715
2.3983	4.72	560000	2.2805
2.3983	4.78	568000	2.2744
2.3974	4.85	576000	2.2678
2.3974	4.92	584000	2.2723
2.388	4.99	592000	2.2655
2.388	5.05	600000	2.2716
2.3921	5.12	608000	2.2771
2.3921	5.19	616000	2.2760
2.3963	5.25	624000	2.2806
2.3963	5.32	632000	2.2697
2.3891	5.39	640000	2.2705
2.3891	5.46	648000	2.2708
2.3968	5.52	656000	2.2689
2.3968	5.59	664000	2.2651
2.3951	5.66	672000	2.2766
2.3951	5.73	680000	2.2717
2.3986	5.79	688000	2.2629
2.3986	5.86	696000	2.2624
2.3985	5.93	704000	2.2693
2.3985	6.0	712000	2.2632
2.4009	6.06	720000	2.2715
2.4009	6.13	728000	2.2654
2.4015	6.2	736000	2.2700
2.4015	6.27	744000	2.2673
2.3927	6.33	752000	2.2701
2.3927	6.4	760000	2.2666
2.3941	6.47	768000	2.2585
2.3941	6.53	776000	2.2679
2.393	6.6	784000	2.2624
2.393	6.67	792000	2.2706
2.4025	6.74	800000	2.2785
2.4025	6.8	808000	2.2658
2.3992	6.87	816000	2.2557
2.3992	6.94	824000	2.2581
2.4055	7.01	832000	2.2725
2.4055	7.07	840000	2.2608
2.3965	7.14	848000	2.2717
2.3965	7.21	856000	2.2643
2.4028	7.28	864000	2.2697
2.4028	7.34	872000	2.2691
2.3943	7.41	880000	2.2628
2.3943	7.48	888000	2.2630
2.3918	7.55	896000	2.2691
2.3918	7.61	904000	2.2778
2.3897	7.68	912000	2.2577
2.3897	7.75	920000	2.2690
2.3996	7.81	928000	2.2631
2.3996	7.88	936000	2.2606
2.4016	7.95	944000	2.2742
2.4016	8.02	952000	2.2705
2.3989	8.08	960000	2.2694
2.3989	8.15	968000	2.2676
2.3989	8.22	976000	2.2659
2.3989	8.29	984000	2.2676
2.3995	8.35	992000	2.2752
2.3995	8.42	1000000	2.2760
2.3958	8.49	1008000	2.2779
2.3958	8.56	1016000	2.2626
2.3962	8.62	1024000	2.2646
2.3962	8.69	1032000	2.2645
2.3966	8.76	1040000	2.2603
2.3966	8.83	1048000	2.2549
2.3934	8.89	1056000	2.2669
2.3934	8.96	1064000	2.2576
2.3918	9.03	1072000	2.2707
2.3918	9.09	1080000	2.2618
2.401	9.16	1088000	2.2680
2.401	9.23	1096000	2.2721
2.3938	9.3	1104000	2.2637
2.3938	9.36	1112000	2.2657
2.3982	9.43	1120000	2.2576
2.3982	9.5	1128000	2.2633
2.4006	9.57	1136000	2.2668
2.4006	9.63	1144000	2.2660
2.3971	9.7	1152000	2.2659
2.3971	9.77	1160000	2.2723
2.4004	9.84	1168000	2.2627
2.4004	9.9	1176000	2.2708
2.3903	9.97	1184000	2.2576
2.3903	10.04	1192000	2.2625
2.3909	10.11	1200000	2.2543
2.3909	10.17	1208000	2.2595
2.4004	10.24	1216000	2.2561
2.4004	10.31	1224000	2.2607
2.3964	10.37	1232000	2.2606
2.3964	10.44	1240000	2.2635
2.4007	10.51	1248000	2.2623
2.4007	10.58	1256000	2.2696
2.3993	10.64	1264000	2.2700
2.3993	10.71	1272000	2.2731
2.4048	10.78	1280000	2.2701
2.4048	10.85	1288000	2.2701
2.3936	10.91	1296000	2.2706
2.3936	10.98	1304000	2.2596
2.3951	11.05	1312000	2.2812
2.3951	11.12	1320000	2.2523
2.39	11.18	1328000	2.2596
2.39	11.25	1336000	2.2723
2.393	11.32	1344000	2.2696
2.393	11.39	1352000	2.2614
2.3915	11.45	1360000	2.2687
2.3915	11.52	1368000	2.2567
2.405	11.59	1376000	2.2717
2.405	11.65	1384000	2.2733
2.3898	11.72	1392000	2.2680
2.3898	11.79	1400000	2.2627
2.3956	11.86	1408000	2.2689
2.3956	11.92	1416000	2.2669
2.4041	11.99	1424000	2.2610
2.4041	12.06	1432000	2.2689
2.3968	12.13	1440000	2.2749
2.3968	12.19	1448000	2.2640
2.4048	12.26	1456000	2.2602
2.4048	12.33	1464000	2.2698
2.4025	12.4	1472000	2.2545
2.4025	12.46	1480000	2.2685
2.3977	12.53	1488000	2.2623
2.3977	12.6	1496000	2.2679
2.3965	12.67	1504000	2.2505
2.3965	12.73	1512000	2.2708
2.3945	12.8	1520000	2.2655
2.3945	12.87	1528000	2.2672
2.3957	12.93	1536000	2.2698
2.3957	13.0	1544000	2.2661
2.3951	13.07	1552000	2.2635
2.3951	13.14	1560000	2.2597
2.4005	13.2	1568000	2.2575
2.4005	13.27	1576000	2.2648
2.394	13.34	1584000	2.2746
2.394	13.41	1592000	2.2722
2.4016	13.47	1600000	2.2567
2.4016	13.54	1608000	2.2599
2.392	13.61	1616000	2.2588
2.392	13.68	1624000	2.2644
2.3936	13.74	1632000	2.2668
2.3936	13.81	1640000	2.2447
2.3954	13.88	1648000	2.2502
2.3954	13.95	1656000	2.2737
2.3901	14.01	1664000	2.2701
2.3901	14.08	1672000	2.2632
2.3963	14.15	1680000	2.2661
2.3963	14.21	1688000	2.2628
2.4005	14.28	1696000	2.2606
2.4005	14.35	1704000	2.2578
2.3877	14.42	1712000	2.2674
2.3877	14.48	1720000	2.2631
2.3958	14.55	1728000	2.2675
2.3958	14.62	1736000	2.2752
2.3858	14.69	1744000	2.2623
2.3858	14.75	1752000	2.2577
2.403	14.82	1760000	2.2512
2.403	14.89	1768000	2.2610
2.3969	14.96	1776000	2.2597
2.3969	15.02	1784000	2.2748
2.4016	15.09	1792000	2.2632
2.4016	15.16	1800000	2.2650
2.4018	15.23	1808000	2.2669
2.4018	15.29	1816000	2.2525
2.3954	15.36	1824000	2.2497
2.3954	15.43	1832000	2.2744
2.396	15.49	1840000	2.2673
2.396	15.56	1848000	2.2637
2.3951	15.63	1856000	2.2615
2.3951	15.7	1864000	2.2644
2.4017	15.76	1872000	2.2656
2.4017	15.83	1880000	2.2682
2.3962	15.9	1888000	2.2592
2.3962	15.97	1896000	2.2643
2.3996	16.03	1904000	2.2648
2.3996	16.1	1912000	2.2706
2.3994	16.17	1920000	2.2700
2.3994	16.24	1928000	2.2627
2.3976	16.3	1936000	2.2592
2.3976	16.37	1944000	2.2606
2.3971	16.44	1952000	2.2588
2.3971	16.51	1960000	2.2607
2.3991	16.57	1968000	2.2692
2.3991	16.64	1976000	2.2548
2.3952	16.71	1984000	2.2572
2.3952	16.77	1992000	2.2626
2.4002	16.84	2000000	2.2680
2.4002	16.91	2008000	2.2690
2.3937	16.98	2016000	2.2523
2.3937	17.04	2024000	2.2700
2.3999	17.11	2032000	2.2652
2.3999	17.18	2040000	2.2671
2.3891	17.25	2048000	2.2700
2.3891	17.31	2056000	2.2589
2.397	17.38	2064000	2.2626
2.397	17.45	2072000	2.2607
2.3968	17.52	2080000	2.2663
2.3968	17.58	2088000	2.2637
2.3932	17.65	2096000	2.2623
2.3932	17.72	2104000	2.2673
2.3981	17.79	2112000	2.2547
2.3981	17.85	2120000	2.2598
2.3964	17.92	2128000	2.2690
2.3964	17.99	2136000	2.2619
2.3941	18.05	2144000	2.2558
2.3941	18.12	2152000	2.2659
2.3926	18.19	2160000	2.2552
2.3926	18.26	2168000	2.2671
2.399	18.32	2176000	2.2661
2.399	18.39	2184000	2.2591
2.3941	18.46	2192000	2.2568
2.3941	18.53	2200000	2.2588
2.3975	18.59	2208000	2.2631
2.3975	18.66	2216000	2.2655
2.3884	18.73	2224000	2.2628
2.3884	18.8	2232000	2.2656
2.399	18.86	2240000	2.2644
2.399	18.93	2248000	2.2608
2.4064	19.0	2256000	2.2561
2.4064	19.07	2264000	2.2680
2.3999	19.13	2272000	2.2703
2.3999	19.2	2280000	2.2624
2.398	19.27	2288000	2.2707
2.398	19.33	2296000	2.2646
2.4007	19.4	2304000	2.2659
2.4007	19.47	2312000	2.2710
2.3955	19.54	2320000	2.2720
2.3955	19.6	2328000	2.2569
2.3973	19.67	2336000	2.2641
2.3973	19.74	2344000	2.2633
2.4059	19.81	2352000	2.2622
2.4059	19.87	2360000	2.2539
2.3899	19.94	2368000	2.2665
2.3899	20.01	2376000	2.2629
2.4025	20.08	2384000	2.2551
2.4025	20.14	2392000	2.2546
2.3956	20.21	2400000	2.2620

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0