jun-han/Whisper-squeezeformer-v2

Whisper-squeezeformer-cross-whisper

This model is a fine-tuned version of openai/whisper-small on the LibriSpeech dataset. It achieves the following results on the evaluation set:

• Loss: 0.1914
• Wer: 9.9874

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 20
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 2500
• training_steps: 52500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
4.6622	1.0	2500	3.9000	116.9678
3.9849	2.0	5000	3.7525	129.2358
3.4683	3.0	7500	1.3437	77.1169
0.5254	4.0	10000	0.4425	24.7470
0.2291	5.0	12500	0.3604	19.0867
0.143	6.0	15000	0.3413	17.6544
0.0969	7.0	17500	0.3331	16.7510
0.071	8.0	20000	0.3366	16.8214
0.2792	9.0	22500	0.2637	13.9588
0.1737	10.0	25000	0.2469	13.0078
0.1174	11.0	27500	0.2442	11.9560
0.0809	12.0	30000	0.2454	12.0549
0.3352	13.0	32500	0.2290	11.8533
0.249	14.0	35000	0.2225	11.5110
0.205	15.0	37500	0.2216	11.4805
0.1802	16.0	40000	0.2225	11.3360
0.2927	17.0	42500	0.2053	10.2975
0.2318	18.0	45000	0.1998	10.2994
0.2327	19.0	47500	0.1936	10.0369
0.2039	20.0	50000	0.1917	9.9646
0.1952	21.0	52500	0.1914	9.9874

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.0
• Datasets 3.1.0
• Tokenizers 0.20.0