asr-africa/mms-1B_all_BIG_C_Bemba_20hr_v1

facebook/mms-1b-all

This model is a fine-tuned version of facebook/mms-1b-all on the BIG_C dataset. It achieves the following results on the evaluation set:

• Loss: 0.3933
• Model Preparation Time: 0.011
• Wer: 0.4507
• Cer: 0.0853

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
3.8696	1.0	616	2.7924	0.011	1.0	0.8563
1.157	2.0	1232	0.6754	0.011	0.5676	0.1461
0.7288	3.0	1848	0.6478	0.011	0.5147	0.1388
0.6804	4.0	2464	0.5980	0.011	0.5267	0.1467
0.6479	5.0	3080	0.5938	0.011	0.5048	0.1529
0.6317	6.0	3696	0.5721	0.011	0.4954	0.1297
0.6212	7.0	4312	0.5598	0.011	0.4998	0.1345
0.6035	8.0	4928	0.5633	0.011	0.4688	0.1253
0.5966	9.0	5544	0.5533	0.011	0.4690	0.1241
0.5847	10.0	6160	0.5574	0.011	0.4727	0.1246
0.5769	11.0	6776	0.5475	0.011	0.4790	0.1325
0.5684	12.0	7392	0.5766	0.011	0.4643	0.1230
0.5599	13.0	8008	0.5398	0.011	0.4587	0.1216
0.5492	14.0	8624	0.5394	0.011	0.4682	0.1241
0.5456	15.0	9240	0.5402	0.011	0.4560	0.1213
0.5386	16.0	9856	0.5550	0.011	0.4514	0.1206
0.5291	17.0	10472	0.5410	0.011	0.4566	0.1218
0.524	18.0	11088	0.5394	0.011	0.4595	0.1280
0.516	19.0	11704	0.5617	0.011	0.4458	0.1195
0.5096	20.0	12320	0.5491	0.011	0.4483	0.1189
0.5058	21.0	12936	0.5486	0.011	0.4529	0.1220
0.499	22.0	13552	0.5690	0.011	0.4445	0.1177
0.4939	23.0	14168	0.5415	0.011	0.4450	0.1194
0.4846	24.0	14784	0.5443	0.011	0.4547	0.1218
0.4787	25.0	15400	0.5520	0.011	0.4459	0.1199
0.4741	26.0	16016	0.5616	0.011	0.4482	0.1181
0.467	27.0	16632	0.5649	0.011	0.4433	0.1178
0.4635	28.0	17248	0.5424	0.011	0.4453	0.1202
0.4593	29.0	17864	0.5842	0.011	0.4413	0.1174
0.4525	30.0	18480	0.5589	0.011	0.4412	0.1209
0.4464	31.0	19096	0.5692	0.011	0.4418	0.1177
0.4417	32.0	19712	0.5631	0.011	0.4408	0.1194
0.4387	33.0	20328	0.5527	0.011	0.4369	0.1185
0.4316	34.0	20944	0.5634	0.011	0.4389	0.1217
0.4284	35.0	21560	0.5819	0.011	0.4378	0.1178
0.4227	36.0	22176	0.5680	0.011	0.4358	0.1172
0.4176	37.0	22792	0.5704	0.011	0.4387	0.1186
0.4137	38.0	23408	0.5711	0.011	0.4378	0.1176
0.4083	39.0	24024	0.5891	0.011	0.4346	0.1171
0.4054	40.0	24640	0.5768	0.011	0.4421	0.1205
0.4025	41.0	25256	0.5972	0.011	0.4341	0.1173
0.3989	42.0	25872	0.5792	0.011	0.4355	0.1176
0.3943	43.0	26488	0.5916	0.011	0.4358	0.1170
0.3895	44.0	27104	0.6200	0.011	0.4373	0.1165
0.3874	45.0	27720	0.6105	0.011	0.4317	0.1162
0.3841	46.0	28336	0.5997	0.011	0.4359	0.1169
0.3792	47.0	28952	0.5925	0.011	0.4342	0.1182
0.3762	48.0	29568	0.6308	0.011	0.4369	0.1163
0.3739	49.0	30184	0.6042	0.011	0.4358	0.1185
0.3687	50.0	30800	0.6019	0.011	0.4364	0.1171
0.3663	51.0	31416	0.6084	0.011	0.4387	0.1185
0.3626	52.0	32032	0.5989	0.011	0.4398	0.1190
0.3601	53.0	32648	0.6050	0.011	0.4393	0.1180
0.357	54.0	33264	0.6059	0.011	0.4389	0.1188
0.3548	55.0	33880	0.6095	0.011	0.4375	0.1183

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.1