All-mms1ball-Dec1

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2261
• Wer: 0.5776

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 3
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
75.7763	0.0050	100	4.0652	1.0
2.6846	0.0101	200	2.1046	0.8644
2.0892	0.0151	300	2.0278	0.7559
2.0608	0.0202	400	1.9999	0.7563
2.2366	0.0252	500	1.8535	0.7402
1.9112	0.0302	600	1.9858	0.7770
1.9297	0.0353	700	1.8104	0.7605
1.8322	0.0403	800	1.7293	0.7351
1.9999	0.0454	900	1.8046	0.7471
1.7927	0.0504	1000	1.6266	0.7112
1.758	0.0554	1100	1.5815	0.7251
1.802	0.0605	1200	1.7270	0.7197
1.8543	0.0655	1300	1.6053	0.7029
1.8065	0.0706	1400	1.5376	0.6837
1.7766	0.0756	1500	1.5990	0.7104
1.7397	0.0806	1600	1.5854	0.7021
1.6967	0.0857	1700	1.5414	0.7214
1.6538	0.0907	1800	1.6067	0.7237
1.7187	0.0958	1900	1.5579	0.7087
1.8067	0.1008	2000	1.7592	0.6927
1.6497	0.1058	2100	1.6402	0.7268
1.8631	0.1109	2200	1.4714	0.6692
1.6328	0.1159	2300	1.5083	0.6820
1.7545	0.1210	2400	1.5050	0.7249
1.6864	0.1260	2500	1.4886	0.7038
1.7195	0.1310	2600	1.5079	0.6585
1.6702	0.1361	2700	1.4524	0.6896
1.6112	0.1411	2800	1.5310	0.6637
1.7391	0.1462	2900	1.7407	0.7562
1.8044	0.1512	3000	1.5270	0.6762
1.6354	0.1562	3100	1.4674	0.6870
1.6693	0.1613	3200	1.4190	0.6663
1.601	0.1663	3300	1.4157	0.6634
1.4786	0.1714	3400	1.5873	0.6997
1.6173	0.1764	3500	1.4899	0.6713
1.6289	0.1815	3600	1.5515	0.6878
1.5271	0.1865	3700	1.5557	0.6821
1.5864	0.1915	3800	1.5196	0.6830
1.6389	0.1966	3900	1.6335	0.6787
1.737	0.2016	4000	1.4266	0.6589
1.4226	0.2067	4100	1.5183	0.6574
1.7838	0.2117	4200	1.5056	0.6927
1.7542	0.2167	4300	1.5173	0.6726
1.4174	0.2218	4400	1.5330	0.6743
1.7392	0.2268	4500	1.4587	0.6589
1.6215	0.2319	4600	1.4066	0.6607
1.5882	0.2369	4700	1.4013	0.6430
1.5614	0.2419	4800	1.4256	0.6366
1.6021	0.2470	4900	1.5503	0.6450
1.6551	0.2520	5000	1.5671	0.6685
1.6113	0.2571	5100	1.4772	0.6682
1.6276	0.2621	5200	1.4259	0.6838
1.4248	0.2671	5300	1.4761	0.6097
1.6195	0.2722	5400	1.3756	0.6340
1.4586	0.2772	5500	1.4569	0.6381
1.5018	0.2823	5600	1.3874	0.6060
1.5791	0.2873	5700	1.3675	0.6143
1.478	0.2923	5800	1.3716	0.6160
1.3886	0.2974	5900	1.3523	0.6084
1.4822	0.3024	6000	1.4189	0.6206
1.3936	0.3075	6100	1.5999	0.6650
1.7154	0.3125	6200	1.4897	0.6438
1.4976	0.3175	6300	1.3664	0.5995
1.5092	0.3226	6400	1.4601	0.6162
1.4872	0.3276	6500	1.3488	0.6119
1.6109	0.3327	6600	1.3319	0.6050
1.529	0.3377	6700	1.3900	0.6207
1.4621	0.3427	6800	1.5310	0.6547
1.5824	0.3478	6900	1.5412	0.6198
1.5586	0.3528	7000	1.3602	0.6082
1.4643	0.3579	7100	1.2699	0.5903
1.4514	0.3629	7200	1.3596	0.6034
1.3783	0.3679	7300	1.5230	0.6452
1.4566	0.3730	7400	1.3042	0.5975
1.6092	0.3780	7500	1.4030	0.6238
1.378	0.3831	7600	1.3766	0.5873
1.3222	0.3881	7700	1.4285	0.6241
1.6084	0.3931	7800	1.2604	0.6030
1.3567	0.3982	7900	1.3863	0.5975
1.3939	0.4032	8000	1.2574	0.5806
1.4154	0.4083	8100	1.2763	0.5825
1.5155	0.4133	8200	1.2933	0.6195
1.4151	0.4183	8300	1.2543	0.5926
1.4554	0.4234	8400	1.3933	0.6348
1.4414	0.4284	8500	1.3746	0.6122
1.6169	0.4335	8600	1.4666	0.5990
1.4456	0.4385	8700	1.2792	0.6025
1.5	0.4435	8800	1.2372	0.6242
1.3626	0.4486	8900	1.2482	0.5949
1.4589	0.4536	9000	1.3655	0.5937
1.4927	0.4587	9100	1.2873	0.5987
1.4479	0.4637	9200	1.3235	0.5998
1.3844	0.4688	9300	1.2443	0.6015
1.4684	0.4738	9400	1.2813	0.5946
1.3035	0.4788	9500	1.4207	0.6015
1.5696	0.4839	9600	1.2985	0.6050
1.3457	0.4889	9700	1.2844	0.5656
1.4152	0.4940	9800	1.2718	0.6010
1.5372	0.4990	9900	1.4137	0.5873
1.3527	0.5040	10000	1.3788	0.5740
1.3174	0.5091	10100	1.3241	0.5668
1.4634	0.5141	10200	1.3513	0.6071
1.3983	0.5192	10300	1.2526	0.5718
1.4222	0.5242	10400	1.2591	0.5957
1.4156	0.5292	10500	1.2508	0.6071
1.4235	0.5343	10600	1.2258	0.5842
1.5522	0.5393	10700	1.2606	0.6380
1.5539	0.5444	10800	1.3713	0.6309
1.2924	0.5494	10900	1.2261	0.5776

Framework versions

• Transformers 4.43.4
• Pytorch 2.4.1
• Datasets 3.0.0
• Tokenizers 0.19.1