mms-1B_all_BIG_C-AMMI-BEMBA_SPEECH_CORPUS_Bemba_189hr_v1

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2920
• Wer: 0.3547
• Cer: 0.0681

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.8602	0.9999	5951	0.3814	0.4324	0.0916
0.5204	1.9999	11902	0.3590	0.4118	0.0873
0.4994	2.9999	17853	0.3547	0.4091	0.0851
0.4859	3.9999	23804	0.3570	0.4039	0.0861
0.4759	4.9999	29755	0.3374	0.3769	0.0809
0.4674	5.9999	35706	0.3338	0.3618	0.0793
0.4611	6.9999	41657	0.3304	0.3597	0.0783
0.4553	7.9999	47608	0.3378	0.3545	0.0812
0.4499	8.9999	53559	0.3355	0.3565	0.0779
0.445	9.9999	59510	0.3222	0.3401	0.0769
0.441	10.9999	65461	0.3213	0.3371	0.0758
0.4369	11.9999	71412	0.3241	0.3382	0.0768
0.4327	12.9999	77363	0.3248	0.3301	0.0747
0.4289	13.9999	83314	0.3215	0.3267	0.0748
0.4262	14.9999	89265	0.3195	0.3275	0.0739
0.4231	15.9999	95216	0.3189	0.3352	0.0744
0.4195	16.9999	101167	0.3169	0.3299	0.0748
0.4174	17.9999	107118	0.3218	0.3144	0.0722
0.4137	18.9999	113069	0.3143	0.3173	0.0724
0.4107	19.9999	119020	0.3144	0.3263	0.0734
0.4088	20.9999	124971	0.3194	0.3218	0.0733
0.4057	21.9999	130922	0.3136	0.3178	0.0751
0.4032	22.9999	136873	0.3139	0.3236	0.0743
0.4006	23.9999	142824	0.3215	0.3125	0.0714
0.3982	24.9999	148775	0.3142	0.3210	0.0728
0.3956	25.9999	154726	0.3118	0.3259	0.0736
0.3928	26.9999	160677	0.3083	0.3098	0.0721
0.3906	27.9999	166628	0.3141	0.3124	0.0733
0.3883	28.9999	172579	0.3129	0.3090	0.0709
0.3861	29.9999	178530	0.3159	0.3167	0.0726
0.3831	30.9999	184481	0.3120	0.3158	0.0716
0.3825	31.9999	190432	0.3127	0.3143	0.0717
0.3804	32.9999	196383	0.3178	0.3141	0.0720
0.3777	33.9999	202334	0.3126	0.3135	0.0717
0.3751	34.9999	208285	0.3096	0.3122	0.0722
0.3737	35.9999	214236	0.3114	0.3147	0.0723
0.3723	36.9999	220187	0.3170	0.3140	0.0729

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.3