arun100/whisper-base-id-1

Whisper Base Indonesian

This model is a fine-tuned version of openai/whisper-base on the mozilla-foundation/common_voice_16_0 id dataset. It achieves the following results on the evaluation set:

• Loss: 0.4198
• Wer: 26.6078

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-07
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.8238	4.01	500	0.6400	37.9750
0.6348	9.01	1000	0.5193	32.2477
0.4879	14.0	1500	0.4829	30.7250
0.4518	19.0	2000	0.4645	29.7037
0.4253	23.01	2500	0.4513	28.8757
0.4471	28.01	3000	0.4409	28.0937
0.3713	33.01	3500	0.4347	27.7854
0.3233	38.0	4000	0.4307	27.6382
0.3152	43.0	4500	0.4280	27.5324
0.3152	47.01	5000	0.4245	27.2196
0.333	52.01	5500	0.4227	26.9942
0.257	57.0	6000	0.4217	26.9620
0.25	62.0	6500	0.4214	26.8148
0.2587	66.01	7000	0.4206	26.7550
0.2765	71.01	7500	0.4198	26.6998
0.2664	76.01	8000	0.4198	26.6216
0.223	81.0	8500	0.4199	26.6446
0.2309	86.0	9000	0.4199	26.6538
0.233	90.01	9500	0.4198	26.6078
0.2647	95.01	10000	0.4198	26.6216

Framework versions

• Transformers 4.37.0.dev0
• Pytorch 2.1.2+cu121
• Datasets 2.16.2.dev0
• Tokenizers 0.15.0