mms-1b-all-sw-CV_Fleurs_AMMI_ALFFA-20hrs-v1

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Wer: 0.2229
• Cer: 0.0780

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Val Cer	Val Wer
5.4496	1.0	756	0.1056	0.3067
0.595	2.0	1512	0.0857	0.2488
0.493	3.0	2268	0.0832	0.2436
0.4618	4.0	3024	0.0818	0.2403
0.4422	5.0	3780	0.0809	0.2366
0.4307	6.0	4536	0.0804	0.2359
0.4236	7.0	5292	0.0798	0.2351
0.4153	8.0	6048	0.0796	0.2348
0.4082	9.0	6804	0.0789	0.2329
0.4067	10.0	7560	0.0787	0.2339
0.4002	11.0	8316	0.0784	0.2329
0.3966	12.0	9072	0.0781	0.2328
0.3905	13.0	9828	0.0784	0.2327
0.3865	14.0	10584	0.0778	0.2318
0.3819	15.0	11340	0.0777	0.2320
0.3768	16.0	12096	0.0777	0.2317
0.3716	17.0	12852	0.0770	0.2299
0.3698	18.0	13608	0.0771	0.2284
0.3647	19.0	14364	0.0769	0.2279
0.3651	20.0	15120	0.0769	0.2297
0.361	21.0	15876	0.0769	0.2284
0.3559	22.0	16632	0.0781	0.2298
0.3576	23.0	17388	0.0793	0.2314
0.3486	24.0	18144	0.0761	0.2258
0.3486	25.0	18900	0.0787	0.2304
0.3455	26.0	19656	0.0769	0.2271
0.3451	27.0	20412	0.0767	0.2256
0.3409	28.0	21168	0.0767	0.2247
0.3394	29.0	21924	0.0802	0.2297

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0