Merged-MM-praj

This model is a fine-tuned version of prajjwal1/bert-tiny on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5525
• Accuracy: 0.7777
• F1: 0.8749

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.0	50	0.6929	0.526	0.3813
No log	0.0	100	0.6938	0.48	0.3125
No log	0.01	150	0.6971	0.479	0.3103
No log	0.01	200	0.6948	0.479	0.3103
No log	0.01	250	0.6938	0.479	0.3103
No log	0.01	300	0.6939	0.479	0.3103
No log	0.01	350	0.6927	0.521	0.3587
No log	0.02	400	0.6931	0.501	0.4988
No log	0.02	450	0.6944	0.479	0.3103
0.6942	0.02	500	0.6954	0.479	0.3103
0.6942	0.02	550	0.6960	0.479	0.3103
0.6942	0.02	600	0.6934	0.486	0.3322
0.6942	0.02	650	0.6970	0.479	0.3103
0.6942	0.03	700	0.6929	0.535	0.4767
0.6942	0.03	750	0.6931	0.499	0.4609
0.6942	0.03	800	0.6952	0.479	0.3103
0.6942	0.03	850	0.6933	0.48	0.3160
0.6942	0.03	900	0.6979	0.479	0.3103
0.6942	0.04	950	0.6940	0.479	0.3103
0.6938	0.04	1000	0.6915	0.521	0.3569
0.6938	0.04	1050	0.6942	0.479	0.3103
0.6938	0.04	1100	0.6884	0.519	0.3630
0.6938	0.04	1150	0.6849	0.596	0.5817
0.6938	0.05	1200	0.6849	0.547	0.5131
0.6938	0.05	1250	0.6771	0.568	0.5502
0.6938	0.05	1300	0.6792	0.572	0.5558
0.6938	0.05	1350	0.6889	0.55	0.5161
0.6938	0.05	1400	0.6792	0.59	0.5828
0.6938	0.06	1450	0.6729	0.602	0.5987
0.6781	0.06	1500	0.6702	0.592	0.5822
0.6781	0.06	1550	0.6711	0.578	0.5633
0.6781	0.06	1600	0.6642	0.607	0.6024
0.6781	0.06	1650	0.6624	0.592	0.5819
0.6781	0.07	1700	0.6585	0.595	0.5883
0.6781	0.07	1750	0.6543	0.584	0.5740
0.6781	0.07	1800	0.6452	0.6	0.5926
0.6781	0.07	1850	0.6355	0.615	0.6106
0.6781	0.07	1900	0.6280	0.615	0.6090
0.6781	0.07	1950	0.6209	0.621	0.6139
0.6465	0.08	2000	0.6178	0.632	0.6247
0.6465	0.08	2050	0.6133	0.641	0.6303
0.6465	0.08	2100	0.6132	0.629	0.6218
0.6465	0.08	2150	0.6155	0.63	0.6289
0.6465	0.08	2200	0.5984	0.635	0.6322
0.6465	0.09	2250	0.6065	0.633	0.6102
0.6465	0.09	2300	0.5968	0.629	0.6063
0.6465	0.09	2350	0.5871	0.649	0.6411
0.6465	0.09	2400	0.5824	0.64	0.6218
0.6465	0.09	2450	0.5812	0.643	0.6390
0.6042	0.1	2500	0.5790	0.644	0.6355
0.6042	0.1	2550	0.5744	0.654	0.6507
0.6042	0.1	2600	0.5679	0.641	0.6292
0.6042	0.1	2650	0.5707	0.644	0.6311
0.6042	0.1	2700	0.5707	0.652	0.6439
0.6042	0.11	2750	0.5680	0.661	0.6569
0.6042	0.11	2800	0.5592	0.67	0.6684
0.6042	0.11	2850	0.5557	0.678	0.6758
0.6042	0.11	2900	0.5579	0.671	0.6690
0.6042	0.11	2950	0.5490	0.692	0.6909
0.5834	0.11	3000	0.5474	0.688	0.6858
0.5834	0.12	3050	0.5447	0.696	0.6902
0.5834	0.12	3100	0.5456	0.699	0.6985
0.5834	0.12	3150	0.5592	0.675	0.6628
0.5834	0.12	3200	0.5442	0.69	0.6856
0.5834	0.12	3250	0.5424	0.698	0.6974
0.5834	0.13	3300	0.5464	0.691	0.6907
0.5834	0.13	3350	0.5433	0.693	0.6922
0.5834	0.13	3400	0.5400	0.746	0.7461
0.5834	0.13	3450	0.5406	0.712	0.7091
0.5551	0.13	3500	0.5367	0.738	0.7376
0.5551	0.14	3550	0.5354	0.713	0.7091
0.5551	0.14	3600	0.5377	0.74	0.7400
0.5551	0.14	3650	0.5342	0.751	0.7506
0.5551	0.14	3700	0.5386	0.701	0.6992
0.5551	0.14	3750	0.5395	0.737	0.7368
0.5551	0.15	3800	0.5333	0.733	0.7330
0.5551	0.15	3850	0.5245	0.737	0.7371
0.5551	0.15	3900	0.5236	0.745	0.7451
0.5551	0.15	3950	0.5149	0.741	0.7400
0.5508	0.15	4000	0.5208	0.743	0.7422
0.5508	0.16	4050	0.5109	0.744	0.7440
0.5508	0.16	4100	0.5179	0.742	0.7398
0.5508	0.16	4150	0.5133	0.75	0.7499
0.5508	0.16	4200	0.5110	0.744	0.7416
0.5508	0.16	4250	0.5133	0.749	0.7476
0.5508	0.16	4300	0.5075	0.743	0.7410
0.5508	0.17	4350	0.5108	0.755	0.7544
0.5508	0.17	4400	0.5051	0.747	0.7465
0.5508	0.17	4450	0.5064	0.746	0.7455
0.5362	0.17	4500	0.5030	0.744	0.7441
0.5362	0.17	4550	0.5043	0.748	0.7476
0.5362	0.18	4600	0.5010	0.753	0.7531
0.5362	0.18	4650	0.4988	0.762	0.7616
0.5362	0.18	4700	0.4999	0.755	0.7548
0.5362	0.18	4750	0.5159	0.754	0.7529
0.5362	0.18	4800	0.4924	0.764	0.7639
0.5362	0.19	4850	0.4935	0.755	0.7549
0.5362	0.19	4900	0.4874	0.76	0.7601
0.5362	0.19	4950	0.4859	0.759	0.7591
0.5226	0.19	5000	0.4901	0.761	0.7610
0.5226	0.19	5050	0.4740	0.779	0.7790
0.5226	0.2	5100	0.4799	0.783	0.7831
0.5226	0.2	5150	0.4833	0.771	0.7698
0.5226	0.2	5200	0.4879	0.759	0.7561
0.5226	0.2	5250	0.4812	0.772	0.7719
0.5226	0.2	5300	0.4825	0.772	0.7715
0.5226	0.2	5350	0.4791	0.775	0.7744
0.5226	0.21	5400	0.4749	0.773	0.7729
0.5226	0.21	5450	0.4691	0.782	0.7811
0.5055	0.21	5500	0.4752	0.78	0.7791
0.5055	0.21	5550	0.4621	0.766	0.7645
0.5055	0.21	5600	0.4628	0.779	0.7790
0.5055	0.22	5650	0.4543	0.776	0.7760
0.5055	0.22	5700	0.4548	0.786	0.7861
0.5055	0.22	5750	0.4578	0.777	0.7763
0.5055	0.22	5800	0.4684	0.778	0.7780
0.5055	0.22	5850	0.4626	0.775	0.7751
0.5055	0.23	5900	0.4714	0.785	0.7850
0.5055	0.23	5950	0.4514	0.79	0.7896
0.4985	0.23	6000	0.4541	0.773	0.7731
0.4985	0.23	6050	0.4587	0.788	0.7876
0.4985	0.23	6100	0.4523	0.787	0.7867
0.4985	0.24	6150	0.4441	0.787	0.7870
0.4985	0.24	6200	0.4529	0.784	0.7841
0.4985	0.24	6250	0.4512	0.784	0.7840
0.4985	0.24	6300	0.4545	0.777	0.7757
0.4985	0.24	6350	0.4399	0.788	0.7874
0.4985	0.25	6400	0.4478	0.794	0.7939
0.4985	0.25	6450	0.4495	0.793	0.7930
0.4937	0.25	6500	0.4454	0.792	0.7913
0.4937	0.25	6550	0.4438	0.795	0.7950
0.4937	0.25	6600	0.4476	0.795	0.7948
0.4937	0.25	6650	0.4448	0.794	0.7939
0.4937	0.26	6700	0.4472	0.791	0.7911
0.4937	0.26	6750	0.4431	0.793	0.7924
0.4937	0.26	6800	0.4434	0.796	0.7958
0.4937	0.26	6850	0.4340	0.802	0.802
0.4937	0.26	6900	0.4502	0.786	0.7848
0.4937	0.27	6950	0.4349	0.797	0.7964
0.4826	0.27	7000	0.4348	0.79	0.7894
0.4826	0.27	7050	0.4321	0.788	0.7875
0.4826	0.27	7100	0.4300	0.787	0.7868
0.4826	0.27	7150	0.4346	0.78	0.7779
0.4826	0.28	7200	0.4246	0.802	0.8020
0.4826	0.28	7250	0.4273	0.793	0.7930
0.4826	0.28	7300	0.4346	0.79	0.7894
0.4826	0.28	7350	0.4358	0.789	0.7887
0.4826	0.28	7400	0.4368	0.788	0.7871
0.4826	0.29	7450	0.4426	0.784	0.7841
0.4756	0.29	7500	0.4312	0.802	0.8019
0.4756	0.29	7550	0.4303	0.795	0.7944
0.4756	0.29	7600	0.4391	0.792	0.7916
0.4756	0.29	7650	0.4325	0.793	0.7922
0.4756	0.29	7700	0.4283	0.793	0.7920
0.4756	0.3	7750	0.4271	0.799	0.7991

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.0
• Tokenizers 0.15.0