arun100/whisper-base-bn-6

Whisper Base Bengali

This model is a fine-tuned version of arun100/whisper-base-bn-3 on the mozilla-foundation/common_voice_16_0 bn dataset. It achieves the following results on the evaluation set:

• Loss: 0.2078
• Wer: 28.8185

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-07
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 5500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.1568	1.72	500	0.2145	29.8350
0.1507	3.43	1000	0.2132	29.5594
0.1466	5.15	1500	0.2119	29.3576
0.1463	6.86	2000	0.2106	29.2927
0.1426	8.58	2500	0.2098	29.2220
0.139	10.29	3000	0.2093	29.1075
0.1373	12.01	3500	0.2087	29.0878
0.1362	13.72	4000	0.2084	28.9769
0.1333	15.44	4500	0.2081	28.9129
0.1332	17.15	5000	0.2079	28.8945
0.1363	18.87	5500	0.2078	28.8185

Framework versions

• Transformers 4.37.0.dev0
• Pytorch 2.1.2+cu121
• Datasets 2.16.2.dev0
• Tokenizers 0.15.0