2020-Q4-25p-filtered

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2653

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.02	8000	2.5926
2.7864	0.04	16000	2.5071
2.7864	0.07	24000	2.4690
2.5937	0.09	32000	2.4355
2.5937	0.11	40000	2.4125
2.55	0.13	48000	2.4009
2.55	0.16	56000	2.3892
2.5159	0.18	64000	2.3736
2.5159	0.2	72000	2.3713
2.495	0.22	80000	2.3641
2.495	0.25	88000	2.3574
2.4845	0.27	96000	2.3491
2.4845	0.29	104000	2.3485
2.4765	0.31	112000	2.3433
2.4765	0.34	120000	2.3376
2.472	0.36	128000	2.3396
2.472	0.38	136000	2.3326
2.467	0.4	144000	2.3384
2.467	0.43	152000	2.3350
2.46	0.45	160000	2.3263
2.46	0.47	168000	2.3231
2.4593	0.49	176000	2.3223
2.4593	0.52	184000	2.3250
2.4552	0.54	192000	2.3195
2.4552	0.56	200000	2.3236
2.4558	0.58	208000	2.3221
2.4558	0.61	216000	2.3194
2.4487	0.63	224000	2.3225
2.4487	0.65	232000	2.3221
2.4485	0.67	240000	2.3135
2.4485	0.7	248000	2.3109
2.4461	0.72	256000	2.3134
2.4461	0.74	264000	2.3177
2.4513	0.76	272000	2.3102
2.4513	0.79	280000	2.3052
2.4488	0.81	288000	2.3044
2.4488	0.83	296000	2.3117
2.4447	0.85	304000	2.3051
2.4447	0.88	312000	2.3112
2.4485	0.9	320000	2.3064
2.4485	0.92	328000	2.3099
2.4475	0.94	336000	2.3110
2.4475	0.97	344000	2.3014
2.4464	0.99	352000	2.3032
2.4464	1.01	360000	2.3036
2.4478	1.03	368000	2.3050
2.4478	1.06	376000	2.3078
2.4416	1.08	384000	2.3028
2.4416	1.1	392000	2.3017
2.4374	1.12	400000	2.3012
2.4374	1.15	408000	2.3017
2.4406	1.17	416000	2.3043
2.4406	1.19	424000	2.3058
2.4434	1.21	432000	2.2938
2.4434	1.24	440000	2.2971
2.4421	1.26	448000	2.3025
2.4421	1.28	456000	2.2950
2.443	1.3	464000	2.2987
2.443	1.32	472000	2.2949
2.4357	1.35	480000	2.3026
2.4357	1.37	488000	2.2961
2.4366	1.39	496000	2.3003
2.4366	1.41	504000	2.2954
2.4528	1.44	512000	2.2883
2.4528	1.46	520000	2.3000
2.4389	1.48	528000	2.2939
2.4389	1.5	536000	2.2990
2.441	1.53	544000	2.2916
2.441	1.55	552000	2.2906
2.4372	1.57	560000	2.2885
2.4372	1.59	568000	2.3003
2.4379	1.62	576000	2.2988
2.4379	1.64	584000	2.2923
2.4347	1.66	592000	2.2937
2.4347	1.68	600000	2.2958
2.4311	1.71	608000	2.2995
2.4311	1.73	616000	2.2941
2.4437	1.75	624000	2.2949
2.4437	1.77	632000	2.2878
2.4306	1.8	640000	2.2895
2.4306	1.82	648000	2.2930
2.4341	1.84	656000	2.2895
2.4341	1.86	664000	2.2908
2.4333	1.89	672000	2.2842
2.4333	1.91	680000	2.2912
2.4403	1.93	688000	2.2900
2.4403	1.95	696000	2.2862
2.4396	1.98	704000	2.2871
2.4396	2.0	712000	2.2948
2.441	2.02	720000	2.2942
2.441	2.04	728000	2.2828
2.434	2.07	736000	2.2808
2.434	2.09	744000	2.2883
2.4387	2.11	752000	2.2923
2.4387	2.13	760000	2.2848
2.4342	2.16	768000	2.2848
2.4342	2.18	776000	2.2865
2.4389	2.2	784000	2.2885
2.4389	2.22	792000	2.2794
2.4318	2.25	800000	2.2861
2.4318	2.27	808000	2.2876
2.4343	2.29	816000	2.2820
2.4343	2.31	824000	2.2835
2.4335	2.34	832000	2.2788
2.4335	2.36	840000	2.2813
2.4428	2.38	848000	2.2789
2.4428	2.4	856000	2.2858
2.4272	2.43	864000	2.2883
2.4272	2.45	872000	2.2809
2.4331	2.47	880000	2.2880
2.4331	2.49	888000	2.2838
2.4326	2.52	896000	2.2804
2.4326	2.54	904000	2.2831
2.436	2.56	912000	2.2867
2.436	2.58	920000	2.2848
2.435	2.6	928000	2.2871
2.435	2.63	936000	2.2828
2.44	2.65	944000	2.2808
2.44	2.67	952000	2.2853
2.4285	2.69	960000	2.2799
2.4285	2.72	968000	2.2829
2.423	2.74	976000	2.2761
2.423	2.76	984000	2.2768
2.4353	2.78	992000	2.2844
2.4353	2.81	1000000	2.2828
2.4301	2.83	1008000	2.2806
2.4301	2.85	1016000	2.2813
2.4284	2.87	1024000	2.2789
2.4284	2.9	1032000	2.2770
2.4252	2.92	1040000	2.2763
2.4252	2.94	1048000	2.2763
2.4289	2.96	1056000	2.2779
2.4289	2.99	1064000	2.2812
2.4349	3.01	1072000	2.2881
2.4349	3.03	1080000	2.2805
2.4365	3.05	1088000	2.2758
2.4365	3.08	1096000	2.2733
2.4274	3.1	1104000	2.2842
2.4274	3.12	1112000	2.2808
2.4326	3.14	1120000	2.2753
2.4326	3.17	1128000	2.2792
2.4244	3.19	1136000	2.2788
2.4244	3.21	1144000	2.2824
2.4285	3.23	1152000	2.2800
2.4285	3.26	1160000	2.2784
2.4371	3.28	1168000	2.2675
2.4371	3.3	1176000	2.2740
2.4273	3.32	1184000	2.2805
2.4273	3.35	1192000	2.2849
2.4359	3.37	1200000	2.2808
2.4359	3.39	1208000	2.2791
2.4303	3.41	1216000	2.2730
2.4303	3.44	1224000	2.2732
2.4306	3.46	1232000	2.2785
2.4306	3.48	1240000	2.2764
2.4267	3.5	1248000	2.2740
2.4267	3.53	1256000	2.2789
2.4271	3.55	1264000	2.2774
2.4271	3.57	1272000	2.2768
2.4263	3.59	1280000	2.2796
2.4263	3.62	1288000	2.2759
2.431	3.64	1296000	2.2741
2.431	3.66	1304000	2.2821
2.4273	3.68	1312000	2.2740
2.4273	3.71	1320000	2.2713
2.4371	3.73	1328000	2.2704
2.4371	3.75	1336000	2.2734
2.4273	3.77	1344000	2.2746
2.4273	3.8	1352000	2.2840
2.4246	3.82	1360000	2.2764
2.4246	3.84	1368000	2.2740
2.4308	3.86	1376000	2.2730
2.4308	3.88	1384000	2.2751
2.4341	3.91	1392000	2.2777
2.4341	3.93	1400000	2.2679
2.4266	3.95	1408000	2.2777
2.4266	3.97	1416000	2.2783
2.4344	4.0	1424000	2.2743
2.4344	4.02	1432000	2.2691
2.431	4.04	1440000	2.2714
2.431	4.06	1448000	2.2694
2.4296	4.09	1456000	2.2749
2.4296	4.11	1464000	2.2810
2.4265	4.13	1472000	2.2744
2.4265	4.15	1480000	2.2714
2.4266	4.18	1488000	2.2733
2.4266	4.2	1496000	2.2790
2.4253	4.22	1504000	2.2766
2.4253	4.24	1512000	2.2764
2.4303	4.27	1520000	2.2692
2.4303	4.29	1528000	2.2684
2.4373	4.31	1536000	2.2752
2.4373	4.33	1544000	2.2701
2.4346	4.36	1552000	2.2758
2.4346	4.38	1560000	2.2727
2.4294	4.4	1568000	2.2753
2.4294	4.42	1576000	2.2687
2.439	4.45	1584000	2.2776
2.439	4.47	1592000	2.2746
2.4337	4.49	1600000	2.2731
2.4337	4.51	1608000	2.2722
2.4273	4.54	1616000	2.2703
2.4273	4.56	1624000	2.2802
2.4275	4.58	1632000	2.2707
2.4275	4.6	1640000	2.2707
2.4201	4.63	1648000	2.2686
2.4201	4.65	1656000	2.2707
2.4319	4.67	1664000	2.2740
2.4319	4.69	1672000	2.2697
2.4314	4.72	1680000	2.2747
2.4314	4.74	1688000	2.2694
2.4242	4.76	1696000	2.2732
2.4242	4.78	1704000	2.2726
2.4302	4.81	1712000	2.2704
2.4302	4.83	1720000	2.2755
2.4375	4.85	1728000	2.2701
2.4375	4.87	1736000	2.2720
2.4305	4.9	1744000	2.2698
2.4305	4.92	1752000	2.2721
2.4353	4.94	1760000	2.2752
2.4353	4.96	1768000	2.2763
2.4274	4.99	1776000	2.2747
2.4274	5.01	1784000	2.2776
2.4234	5.03	1792000	2.2706
2.4234	5.05	1800000	2.2719
2.4304	5.08	1808000	2.2667
2.4304	5.1	1816000	2.2762
2.4308	5.12	1824000	2.2757
2.4308	5.14	1832000	2.2712
2.4342	5.16	1840000	2.2676
2.4342	5.19	1848000	2.2738
2.4342	5.21	1856000	2.2755
2.4342	5.23	1864000	2.2741
2.4329	5.25	1872000	2.2734
2.4329	5.28	1880000	2.2714
2.4306	5.3	1888000	2.2722
2.4306	5.32	1896000	2.2702
2.4302	5.34	1904000	2.2761
2.4302	5.37	1912000	2.2748
2.4303	5.39	1920000	2.2763
2.4303	5.41	1928000	2.2731
2.4234	5.43	1936000	2.2676
2.4234	5.46	1944000	2.2750
2.4349	5.48	1952000	2.2769
2.4349	5.5	1960000	2.2728
2.4295	5.52	1968000	2.2750
2.4295	5.55	1976000	2.2702
2.428	5.57	1984000	2.2729
2.428	5.59	1992000	2.2707
2.4336	5.61	2000000	2.2774
2.4336	5.64	2008000	2.2735
2.4332	5.66	2016000	2.2634
2.4332	5.68	2024000	2.2679
2.4342	5.7	2032000	2.2753
2.4342	5.73	2040000	2.2719
2.4279	5.75	2048000	2.2711
2.4279	5.77	2056000	2.2778
2.4281	5.79	2064000	2.2693
2.4281	5.82	2072000	2.2715
2.4246	5.84	2080000	2.2674
2.4246	5.86	2088000	2.2700
2.4235	5.88	2096000	2.2703
2.4235	5.91	2104000	2.2723
2.4388	5.93	2112000	2.2683
2.4388	5.95	2120000	2.2712
2.431	5.97	2128000	2.2739
2.431	6.0	2136000	2.2757
2.4329	6.02	2144000	2.2785
2.4329	6.04	2152000	2.2721
2.4266	6.06	2160000	2.2745
2.4266	6.09	2168000	2.2738
2.4255	6.11	2176000	2.2735
2.4255	6.13	2184000	2.2667
2.4263	6.15	2192000	2.2766
2.4263	6.18	2200000	2.2754
2.4388	6.2	2208000	2.2694
2.4388	6.22	2216000	2.2675
2.4293	6.24	2224000	2.2699
2.4293	6.27	2232000	2.2712
2.428	6.29	2240000	2.2707
2.428	6.31	2248000	2.2732
2.4247	6.33	2256000	2.2752
2.4247	6.36	2264000	2.2703
2.4272	6.38	2272000	2.2690
2.4272	6.4	2280000	2.2775
2.4297	6.42	2288000	2.2680
2.4297	6.45	2296000	2.2712
2.4268	6.47	2304000	2.2815
2.4268	6.49	2312000	2.2697
2.4248	6.51	2320000	2.2794
2.4248	6.53	2328000	2.2722
2.4285	6.56	2336000	2.2686
2.4285	6.58	2344000	2.2741
2.4318	6.6	2352000	2.2679
2.4318	6.62	2360000	2.2723
2.4269	6.65	2368000	2.2741
2.4269	6.67	2376000	2.2739
2.4275	6.69	2384000	2.2744
2.4275	6.71	2392000	2.2765
2.4259	6.74	2400000	2.2788

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0