mms-zeroshot-300m-genbed-combined-model

This model is a fine-tuned version of mms-meta/mms-zeroshot-300m on the GENBED - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.2768
• Wer: 0.4294

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.2747	200	2.2031	1.0
No log	0.5495	400	0.4235	0.6042
2.7468	0.8242	600	0.3791	0.5722
2.7468	1.0989	800	0.3588	0.5632
0.5446	1.3736	1000	0.3489	0.5446
0.5446	1.6484	1200	0.3406	0.5458
0.5446	1.9231	1400	0.3326	0.5183
0.4932	2.1978	1600	0.3235	0.5237
0.4932	2.4725	1800	0.3202	0.5058
0.4644	2.7473	2000	0.3204	0.504
0.4644	3.0220	2200	0.3144	0.4910
0.4644	3.2967	2400	0.3076	0.4957
0.4505	3.5714	2600	0.3039	0.4751
0.4505	3.8462	2800	0.3027	0.4785
0.433	4.1209	3000	0.3013	0.4682
0.433	4.3956	3200	0.2961	0.4765
0.433	4.6703	3400	0.2914	0.468
0.4203	4.9451	3600	0.2909	0.4656
0.4203	5.2198	3800	0.2946	0.4508
0.4042	5.4945	4000	0.2901	0.4434
0.4042	5.7692	4200	0.2900	0.4463
0.4042	6.0440	4400	0.2840	0.4449
0.3962	6.3187	4600	0.2820	0.4448
0.3962	6.5934	4800	0.2771	0.4326
0.3881	6.8681	5000	0.2768	0.4294
0.3881	7.1429	5200	0.2787	0.4385
0.3881	7.4176	5400	0.2795	0.4309
0.3784	7.6923	5600	0.2803	0.4357

Framework versions

• Transformers 4.46.0.dev0
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0